[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:40:17 dist-test-slave-xxxx krb5kdc[6535](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:40:17 dist-test-slave-xxxx krb5kdc[6535](info): set up 2 sockets
Sep 26 11:40:17 dist-test-slave-xxxx krb5kdc[6535](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260926 11:40:18.471475 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:40:18 dist-test-slave-xxxx krb5kdc[6535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:40:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:40:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:40:18.590248 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:43469
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:40597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:43469
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:18.698580 6548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:18.698822 6548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:18.698908 6548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:18.702961 6548 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:40:18.703011 6548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:18.703028 6548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:18.703042 6548 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:40:18.703059 6548 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:40:18.708258 6548 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:40597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:43469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:43469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:18.709273 6548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:18.710233 6548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:18.718714 6554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:18.718779 6548 server_base.cc:1034] running on GCE node
W20260926 11:40:18.719036 6553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:18.719075 6556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:18.719775 6548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:18.721179 6548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:18.722339 6548 hybrid_clock.cc:648] HybridClock initialized: now 1790422818722305 us; error 52 us; skew 500 ppm
Sep 26 11:40:18 dist-test-slave-xxxx krb5kdc[6535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422818, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:18.725759 6548 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:40:18.726883 6548 webserver.cc:460] Webserver started at http://127.6.96.62:43667/ using document root <none> and password file <none>
I20260926 11:40:18.727456 6548 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:18.727525 6548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:18.727726 6548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:18.729794 6548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "888fbcde4c5a4913bf820e283168f700"
format_stamp: "Formatted at 2026-09-26 11:40:18 on dist-test-slave-xxxx"
server_key: "639e835ad892efc5828d987e82fac7ad"
server_key_iv: "90335256176de8e3b35f4a234e33aa3f"
server_key_version: "encryptionkey@0"
I20260926 11:40:18.730341 6548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "888fbcde4c5a4913bf820e283168f700"
format_stamp: "Formatted at 2026-09-26 11:40:18 on dist-test-slave-xxxx"
server_key: "639e835ad892efc5828d987e82fac7ad"
server_key_iv: "90335256176de8e3b35f4a234e33aa3f"
server_key_version: "encryptionkey@0"
I20260926 11:40:18.734077 6548 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 11:40:18.736475 6563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:18.737663 6548 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 11:40:18.737766 6548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "888fbcde4c5a4913bf820e283168f700"
format_stamp: "Formatted at 2026-09-26 11:40:18 on dist-test-slave-xxxx"
server_key: "639e835ad892efc5828d987e82fac7ad"
server_key_iv: "90335256176de8e3b35f4a234e33aa3f"
server_key_version: "encryptionkey@0"
I20260926 11:40:18.737862 6548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:18.757517 6548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:18.760392 6548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:18.760555 6548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:18.767262 6548 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:43469
I20260926 11:40:18.767272 6615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:43469 every 8 connection(s)
I20260926 11:40:18.768187 6548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:40:18.770887 6616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:18.776126 6616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 888fbcde4c5a4913bf820e283168f700: Bootstrap starting.
I20260926 11:40:18.776427 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 6548
I20260926 11:40:18.776558 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:40:18.777745 6528 external_mini_cluster.cc:1428] Setting key 49b4a970f2b8c5efa8a7b254a8d0ed87
I20260926 11:40:18.778509 6616 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 888fbcde4c5a4913bf820e283168f700: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:18.779223 6616 log.cc:824] T 00000000000000000000000000000000 P 888fbcde4c5a4913bf820e283168f700: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:18.781426 6616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 888fbcde4c5a4913bf820e283168f700: No bootstrap required, opened a new log
I20260926 11:40:18.785341 6616 raft_consensus.cc:348] T 00000000000000000000000000000000 P 888fbcde4c5a4913bf820e283168f700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888fbcde4c5a4913bf820e283168f700" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43469 } }
I20260926 11:40:18.785578 6616 raft_consensus.cc:374] T 00000000000000000000000000000000 P 888fbcde4c5a4913bf820e283168f700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:18.785642 6616 raft_consensus.cc:729] T 00000000000000000000000000000000 P 888fbcde4c5a4913bf820e283168f700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 888fbcde4c5a4913bf820e283168f700, State: Initialized, Role: FOLLOWER
I20260926 11:40:18.786072 6616 consensus_queue.cc:260] T 00000000000000000000000000000000 P 888fbcde4c5a4913bf820e283168f700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888fbcde4c5a4913bf820e283168f700" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43469 } }
I20260926 11:40:18.786206 6616 raft_consensus.cc:388] T 00000000000000000000000000000000 P 888fbcde4c5a4913bf820e283168f700 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:40:18.786266 6616 raft_consensus.cc:482] T 00000000000000000000000000000000 P 888fbcde4c5a4913bf820e283168f700 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:40:18.786360 6616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 888fbcde4c5a4913bf820e283168f700 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:18.787350 6616 raft_consensus.cc:504] T 00000000000000000000000000000000 P 888fbcde4c5a4913bf820e283168f700 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888fbcde4c5a4913bf820e283168f700" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43469 } }
I20260926 11:40:18.787667 6616 leader_election.cc:302] T 00000000000000000000000000000000 P 888fbcde4c5a4913bf820e283168f700 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 888fbcde4c5a4913bf820e283168f700; no voters:
I20260926 11:40:18.787953 6616 leader_election.cc:288] T 00000000000000000000000000000000 P 888fbcde4c5a4913bf820e283168f700 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:40:18.788039 6621 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 888fbcde4c5a4913bf820e283168f700 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:18.788192 6621 raft_consensus.cc:686] T 00000000000000000000000000000000 P 888fbcde4c5a4913bf820e283168f700 [term 1 LEADER]: Becoming Leader. State: Replica: 888fbcde4c5a4913bf820e283168f700, State: Running, Role: LEADER
I20260926 11:40:18.788492 6621 consensus_queue.cc:237] T 00000000000000000000000000000000 P 888fbcde4c5a4913bf820e283168f700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888fbcde4c5a4913bf820e283168f700" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43469 } }
I20260926 11:40:18.789238 6616 sys_catalog.cc:564] T 00000000000000000000000000000000 P 888fbcde4c5a4913bf820e283168f700 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:40:18.792059 6623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 888fbcde4c5a4913bf820e283168f700 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "888fbcde4c5a4913bf820e283168f700" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888fbcde4c5a4913bf820e283168f700" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43469 } } }
I20260926 11:40:18.792091 6622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 888fbcde4c5a4913bf820e283168f700 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 888fbcde4c5a4913bf820e283168f700. Latest consensus state: current_term: 1 leader_uuid: "888fbcde4c5a4913bf820e283168f700" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888fbcde4c5a4913bf820e283168f700" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43469 } } }
I20260926 11:40:18.792184 6623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 888fbcde4c5a4913bf820e283168f700 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:18.792196 6622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 888fbcde4c5a4913bf820e283168f700 [sys.catalog]: This master's current role is: LEADER
W20260926 11:40:18.794226 6636 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 888fbcde4c5a4913bf820e283168f700: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 11:40:18.794291 6636 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 11:40:18.794355 6637 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 26 11:40:18 dist-test-slave-xxxx krb5kdc[6535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:18.795157 6637 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:40:18.801859 6637 catalog_manager.cc:1347] Generated new cluster ID: 06416c1aca0d4344ab9fe2997f5790e8
I20260926 11:40:18.801860 6619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:18.780509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56834 (local address 127.6.96.62:43469)
0926 11:40:18.780854 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:18.780868 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:18.782248 (+ 1380us) server_negotiation.cc:407] Connection header received
0926 11:40:18.784954 (+ 2706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:18.784970 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:18.785263 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:18.785580 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:18.787904 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:18.789094 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:18.790707 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:18.791327 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:18.791482 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:18.795921 (+ 4439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:18.795927 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:18.795943 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:18.795970 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:18.797988 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:18.798676 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:18.798682 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:18.798687 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:18.798813 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:18.799283 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:18.799287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:18.799289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:18.799819 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:18.800011 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:18.800616 (+ 605us) server_negotiation.cc:299] Negotiation successful
0926 11:40:18.800882 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":84,"threads_started":1}
I20260926 11:40:18.801942 6637 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:40:18.812485 6637 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:40:18.813618 6637 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:40:18.822012 6637 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 888fbcde4c5a4913bf820e283168f700: Generated new TSK 0
I20260926 11:40:18.822897 6637 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.96.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
I20260926 11:40:18.855986 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:43469
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:40597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:18.963003 6642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:18.963266 6642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:18.963328 6642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:18.967356 6642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:18.967422 6642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:18.967507 6642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:40:18.972775 6642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:40597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:43469
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:18.973841 6642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:18.974869 6642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:18.984181 6648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:18.984182 6650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:18.984205 6647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:18.984346 6642 server_base.cc:1034] running on GCE node
I20260926 11:40:18.984900 6642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:18.985550 6642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:18.986720 6642 hybrid_clock.cc:648] HybridClock initialized: now 1790422818986703 us; error 29 us; skew 500 ppm
Sep 26 11:40:18 dist-test-slave-xxxx krb5kdc[6535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422818, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:18.989300 6642 init.cc:377] Logged in from keytab as kudu/127.6.96.1@KRBTEST.COM (short username kudu)
I20260926 11:40:18.990202 6642 webserver.cc:460] Webserver started at http://127.6.96.1:33993/ using document root <none> and password file <none>
I20260926 11:40:18.990760 6642 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:18.990850 6642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:18.991053 6642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:18.993031 6642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "f218c3f633034f2aaa5bfe8ec4ecd807"
format_stamp: "Formatted at 2026-09-26 11:40:18 on dist-test-slave-xxxx"
server_key: "fe329b09424078f0cdaaa80ec1cd4b1c"
server_key_iv: "d78148687a7bbbb4e3ad97552671dab1"
server_key_version: "encryptionkey@0"
I20260926 11:40:18.993543 6642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "f218c3f633034f2aaa5bfe8ec4ecd807"
format_stamp: "Formatted at 2026-09-26 11:40:18 on dist-test-slave-xxxx"
server_key: "fe329b09424078f0cdaaa80ec1cd4b1c"
server_key_iv: "d78148687a7bbbb4e3ad97552671dab1"
server_key_version: "encryptionkey@0"
I20260926 11:40:18.997076 6642 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260926 11:40:18.999378 6657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:19.000589 6642 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:40:19.000726 6642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "f218c3f633034f2aaa5bfe8ec4ecd807"
format_stamp: "Formatted at 2026-09-26 11:40:18 on dist-test-slave-xxxx"
server_key: "fe329b09424078f0cdaaa80ec1cd4b1c"
server_key_iv: "d78148687a7bbbb4e3ad97552671dab1"
server_key_version: "encryptionkey@0"
I20260926 11:40:19.000819 6642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:19.025743 6642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:19.028765 6642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:19.028955 6642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:19.029539 6642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:19.030368 6642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:19.030416 6642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:19.030467 6642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:19.030495 6642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:19.038892 6642 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:35957
I20260926 11:40:19.038926 6770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:35957 every 8 connection(s)
I20260926 11:40:19.039759 6642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:40:19.042188 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 6642
I20260926 11:40:19.042285 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:40:19.042518 6528 external_mini_cluster.cc:1428] Setting key d418b123686a52dae7808224ebe76136
Sep 26 11:40:19 dist-test-slave-xxxx krb5kdc[6535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422818, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.96.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.2@KRBTEST.COM" created.
I20260926 11:40:19.053236 6619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:19.041558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:50085 (local address 127.6.96.62:43469)
0926 11:40:19.041738 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:19.041743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:19.042572 (+ 829us) server_negotiation.cc:407] Connection header received
0926 11:40:19.044184 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:19.044188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:19.044237 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:19.044319 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:19.045495 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:19.046014 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:19.047176 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:19.047408 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:19.047495 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:19.049811 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:19.049814 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:19.049816 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:19.049838 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:19.051263 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:19.051747 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:19.051751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:19.051752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:19.051819 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:19.052121 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:19.052125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:19.052126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:19.052319 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:19.052450 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:19.052927 (+ 477us) server_negotiation.cc:299] Negotiation successful
0926 11:40:19.053060 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 11:40:19.054042 6773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:19.041955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:43469 (local address 127.6.96.1:50085)
0926 11:40:19.042404 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:19.042454 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:19.043971 (+ 1517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:19.044493 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:19.044500 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:19.044849 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:19.045356 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:19.045371 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:19.046128 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:19.046132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:19.047064 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:19.047073 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:19.047549 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:19.047553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:19.047716 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:19.048286 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:19.048300 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:19.049662 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:19.051375 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:19.051381 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:19.051396 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:19.051660 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:19.051906 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:19.051909 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:19.051911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:19.052039 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:19.052454 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:19.052461 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:19.052701 (+ 240us) client_negotiation.cc:769] Sending connection context
0926 11:40:19.052885 (+ 184us) client_negotiation.cc:240] Negotiation successful
0926 11:40:19.053206 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":98,"threads_started":1}
I20260926 11:40:19.055274 6771 heartbeater.cc:347] Connected to a master server at 127.6.96.62:43469
I20260926 11:40:19.055511 6771 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:19.056011 6771 heartbeater.cc:511] Master 127.6.96.62:43469 requested a full tablet report, sending...
I20260926 11:40:19.057533 6580 ts_manager.cc:194] Registered new tserver with Master: f218c3f633034f2aaa5bfe8ec4ecd807 (127.6.96.1:35957)
I20260926 11:40:19.058614 6580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.1@KRBTEST.COM'} at 127.6.96.1:50085
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
I20260926 11:40:19.067436 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:43469
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:40597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:19.176868 6776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:19.177150 6776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:19.177193 6776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:19.181386 6776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:19.181447 6776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:19.181516 6776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:40:19.186810 6776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:40597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:43469
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:19.187899 6776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:19.188934 6776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:19.198278 6782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:19.198308 6781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:19.198380 6776 server_base.cc:1034] running on GCE node
W20260926 11:40:19.198292 6784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:19.198908 6776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:19.199556 6776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:19.200717 6776 hybrid_clock.cc:648] HybridClock initialized: now 1790422819200681 us; error 47 us; skew 500 ppm
Sep 26 11:40:19 dist-test-slave-xxxx krb5kdc[6535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422819, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:19.203437 6776 init.cc:377] Logged in from keytab as kudu/127.6.96.2@KRBTEST.COM (short username kudu)
I20260926 11:40:19.204259 6776 webserver.cc:460] Webserver started at http://127.6.96.2:35191/ using document root <none> and password file <none>
I20260926 11:40:19.204792 6776 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:19.204850 6776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:19.205013 6776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:19.207067 6776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "f91bdab0ebc8489b90df69554b29ed07"
format_stamp: "Formatted at 2026-09-26 11:40:19 on dist-test-slave-xxxx"
server_key: "252250f451fe5dbdcf5064b93b484cca"
server_key_iv: "a3a73bf8110501098cb08df486d0323a"
server_key_version: "encryptionkey@0"
I20260926 11:40:19.207564 6776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "f91bdab0ebc8489b90df69554b29ed07"
format_stamp: "Formatted at 2026-09-26 11:40:19 on dist-test-slave-xxxx"
server_key: "252250f451fe5dbdcf5064b93b484cca"
server_key_iv: "a3a73bf8110501098cb08df486d0323a"
server_key_version: "encryptionkey@0"
I20260926 11:40:19.211063 6776 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:40:19.213426 6791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:19.214681 6776 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:19.214777 6776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "f91bdab0ebc8489b90df69554b29ed07"
format_stamp: "Formatted at 2026-09-26 11:40:19 on dist-test-slave-xxxx"
server_key: "252250f451fe5dbdcf5064b93b484cca"
server_key_iv: "a3a73bf8110501098cb08df486d0323a"
server_key_version: "encryptionkey@0"
I20260926 11:40:19.214916 6776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:19.242435 6776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:19.245303 6776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:19.245471 6776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:19.246053 6776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:19.246897 6776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:19.246946 6776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:19.246989 6776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:19.247015 6776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:19.254992 6776 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:37365
I20260926 11:40:19.255026 6904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:37365 every 8 connection(s)
I20260926 11:40:19.255825 6776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:40:19.263195 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 6776
I20260926 11:40:19.263316 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:40:19.263605 6528 external_mini_cluster.cc:1428] Setting key 0f087ade7bd47797e57a4e93116266e0
Sep 26 11:40:19 dist-test-slave-xxxx krb5kdc[6535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422819, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:19.269073 6619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:19.257499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:44119 (local address 127.6.96.62:43469)
0926 11:40:19.257625 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:19.257629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:19.258399 (+ 770us) server_negotiation.cc:407] Connection header received
0926 11:40:19.259682 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:19.259685 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:19.259731 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:19.259795 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:19.260937 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:19.261508 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:19.262695 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:19.262941 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:19.263027 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:19.265662 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:19.265665 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:19.265666 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:19.265687 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:19.267217 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:19.267689 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:19.267693 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:19.267694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:19.267766 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:19.268069 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:19.268073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:19.268074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:19.268240 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:19.268320 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:19.268794 (+ 474us) server_negotiation.cc:299] Negotiation successful
0926 11:40:19.268922 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 11:40:19.269904 6907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:19.257855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:43469 (local address 127.6.96.2:44119)
0926 11:40:19.258256 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:19.258296 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:19.259486 (+ 1190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:19.260006 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:19.260013 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:19.260349 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:19.260815 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:19.260829 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:19.261634 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:19.261638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:19.262591 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:19.262601 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:19.263112 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:19.263116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:19.263308 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:19.263890 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:19.263911 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:19.265515 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:19.267325 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:19.267331 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:19.267346 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:19.267595 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:19.267845 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:19.267848 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:19.267850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:19.267986 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:19.268332 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:19.268338 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:19.268578 (+ 240us) client_negotiation.cc:769] Sending connection context
0926 11:40:19.268742 (+ 164us) client_negotiation.cc:240] Negotiation successful
0926 11:40:19.269080 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":99,"threads_started":1}
I20260926 11:40:19.270998 6905 heartbeater.cc:347] Connected to a master server at 127.6.96.62:43469
I20260926 11:40:19.271211 6905 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.6.96.3@KRBTEST.COM; defaulting to no policy
I20260926 11:40:19.271677 6905 heartbeater.cc:511] Master 127.6.96.62:43469 requested a full tablet report, sending...
I20260926 11:40:19.272650 6580 ts_manager.cc:194] Registered new tserver with Master: f91bdab0ebc8489b90df69554b29ed07 (127.6.96.2:37365)
I20260926 11:40:19.273205 6580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.2@KRBTEST.COM'} at 127.6.96.2:44119
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
I20260926 11:40:19.289537 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:43469
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:40597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:19.397553 6910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:19.397809 6910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:19.397883 6910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:19.402117 6910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:19.402184 6910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:19.402261 6910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:40:19.407598 6910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:40597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:43469
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:19.408594 6910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:19.409595 6910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:19.418864 6916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:19.418864 6915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:19.419104 6910 server_base.cc:1034] running on GCE node
W20260926 11:40:19.418874 6918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:19.419478 6910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:19.420097 6910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:19.421263 6910 hybrid_clock.cc:648] HybridClock initialized: now 1790422819421245 us; error 28 us; skew 500 ppm
Sep 26 11:40:19 dist-test-slave-xxxx krb5kdc[6535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422819, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:19.423899 6910 init.cc:377] Logged in from keytab as kudu/127.6.96.3@KRBTEST.COM (short username kudu)
I20260926 11:40:19.424714 6910 webserver.cc:460] Webserver started at http://127.6.96.3:36687/ using document root <none> and password file <none>
I20260926 11:40:19.425220 6910 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:19.425272 6910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:19.425434 6910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:19.427438 6910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "8789d7b14a5d4176a7df93fb00ac1392"
format_stamp: "Formatted at 2026-09-26 11:40:19 on dist-test-slave-xxxx"
server_key: "82b8d5ec9e44c5448c492514a4648731"
server_key_iv: "bf370901fd20c139c508c6c5507f76bd"
server_key_version: "encryptionkey@0"
I20260926 11:40:19.428000 6910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "8789d7b14a5d4176a7df93fb00ac1392"
format_stamp: "Formatted at 2026-09-26 11:40:19 on dist-test-slave-xxxx"
server_key: "82b8d5ec9e44c5448c492514a4648731"
server_key_iv: "bf370901fd20c139c508c6c5507f76bd"
server_key_version: "encryptionkey@0"
I20260926 11:40:19.431658 6910 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 11:40:19.434024 6925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:19.435274 6910 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:40:19.435391 6910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "8789d7b14a5d4176a7df93fb00ac1392"
format_stamp: "Formatted at 2026-09-26 11:40:19 on dist-test-slave-xxxx"
server_key: "82b8d5ec9e44c5448c492514a4648731"
server_key_iv: "bf370901fd20c139c508c6c5507f76bd"
server_key_version: "encryptionkey@0"
I20260926 11:40:19.435494 6910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:19.463979 6910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:19.467248 6910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:19.467404 6910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:19.467958 6910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:19.468766 6910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:19.468813 6910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:19.468865 6910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:19.468887 6910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:19.476980 6910 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:33803
I20260926 11:40:19.476996 7038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:33803 every 8 connection(s)
I20260926 11:40:19.477840 6910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:40:19.485586 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 6910
I20260926 11:40:19.485677 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:40:19.485908 6528 external_mini_cluster.cc:1428] Setting key a892ffc6b46eef6ea6630f3e8e4ead1b
Sep 26 11:40:19 dist-test-slave-xxxx krb5kdc[6535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422819, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:19.491459 6619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:19.479570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:37713 (local address 127.6.96.62:43469)
0926 11:40:19.479702 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:19.479705 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:19.480567 (+ 862us) server_negotiation.cc:407] Connection header received
0926 11:40:19.481927 (+ 1360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:19.481931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:19.481988 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:19.482065 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:19.483300 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:19.483830 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:19.484958 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:19.485176 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:19.485263 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:19.488018 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:19.488023 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:19.488025 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:19.488049 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:19.489334 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:19.489813 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:19.489815 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:19.489816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:19.489887 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:19.490293 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:19.490295 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:19.490297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:19.490536 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:19.490628 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:19.491175 (+ 547us) server_negotiation.cc:299] Negotiation successful
0926 11:40:19.491296 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 11:40:19.492573 7041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:19.479939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:43469 (local address 127.6.96.3:37713)
0926 11:40:19.480403 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:19.480447 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:19.481702 (+ 1255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:19.482250 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:19.482259 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:19.482615 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:19.483152 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:19.483166 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:19.483951 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:19.483955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:19.484832 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:19.484841 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:19.485313 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:19.485317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:19.485478 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:19.486108 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:19.486124 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:19.487858 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:19.489457 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:19.489463 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:19.489479 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:19.489735 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:19.489997 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:19.490000 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:19.490001 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:19.490153 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:19.490637 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:19.490643 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:19.490937 (+ 294us) client_negotiation.cc:769] Sending connection context
0926 11:40:19.491134 (+ 197us) client_negotiation.cc:240] Negotiation successful
0926 11:40:19.491614 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":98,"threads_started":1}
I20260926 11:40:19.493583 7039 heartbeater.cc:347] Connected to a master server at 127.6.96.62:43469
I20260926 11:40:19.493808 7039 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:19.494313 7039 heartbeater.cc:511] Master 127.6.96.62:43469 requested a full tablet report, sending...
I20260926 11:40:19.495287 6580 ts_manager.cc:194] Registered new tserver with Master: 8789d7b14a5d4176a7df93fb00ac1392 (127.6.96.3:33803)
I20260926 11:40:19.495817 6580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.3@KRBTEST.COM'} at 127.6.96.3:37713
I20260926 11:40:19.500144 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 11:40:19 dist-test-slave-xxxx krb5kdc[6535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422819, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 11:40:19 dist-test-slave-xxxx krb5kdc[6535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422819, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.96.1@KRBTEST.COM
I20260926 11:40:19.523336 7046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:19.515060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35662 (local address 127.6.96.1:35957)
0926 11:40:19.515313 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:19.515327 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:19.515350 (+ 23us) server_negotiation.cc:407] Connection header received
0926 11:40:19.515463 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:19.515468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:19.515652 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:19.515768 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:19.516366 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:19.517124 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:19.517876 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:19.518248 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:19.518326 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:19.520533 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:19.520542 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:19.520554 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:19.520576 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:19.521843 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:19.522209 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:19.522214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:19.522215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:19.522285 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:19.522462 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:19.522464 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:19.522466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:19.522677 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:19.522816 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:19.523015 (+ 199us) server_negotiation.cc:299] Negotiation successful
0926 11:40:19.523147 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":72,"threads_started":1}
W20260926 11:40:19.524298 6685 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35662
Sep 26 11:40:19 dist-test-slave-xxxx krb5kdc[6535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 11:40:19 dist-test-slave-xxxx krb5kdc[6535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.1@KRBTEST.COM
I20260926 11:40:19.548317 7046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:19.540804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35664 (local address 127.6.96.1:35957)
0926 11:40:19.540940 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:19.540945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:19.541076 (+ 131us) server_negotiation.cc:407] Connection header received
0926 11:40:19.541216 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:19.541219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:19.541262 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:19.541328 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:19.541991 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:19.542567 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:19.543433 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:19.543661 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:19.543746 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:19.545914 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:19.545917 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:19.545919 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:19.545945 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:19.547121 (+ 1176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:19.547455 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:19.547458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:19.547459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:19.547523 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:19.547696 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:19.547698 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:19.547699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:19.547840 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:19.547926 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:19.548080 (+ 154us) server_negotiation.cc:299] Negotiation successful
0926 11:40:19.548168 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 11:40:19.549206 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 6642
I20260926 11:40:19.553857 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 6776
I20260926 11:40:19.558219 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 6910
I20260926 11:40:19.562544 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 6548
2026-09-26T11:40:19Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2054 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:40:19 dist-test-slave-xxxx krb5kdc[7053](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:40:19 dist-test-slave-xxxx krb5kdc[7053](info): set up 2 sockets
Sep 26 11:40:19 dist-test-slave-xxxx krb5kdc[7053](info): commencing operation
krb5kdc: starting...
W20260926 11:40:20.510289 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:40:20 dist-test-slave-xxxx krb5kdc[7053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:40:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:40:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:40:20.624226 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:41281
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:40061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:41281
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:20.733150 7066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:20.733417 7066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:20.733471 7066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:20.737534 7066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:40:20.737596 7066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:20.737622 7066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:20.737644 7066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:40:20.737673 7066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:40:20.742956 7066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:40061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:41281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:41281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:20.743948 7066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:20.744916 7066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 11:40:20.753216 7066 server_base.cc:1034] running on GCE node
W20260926 11:40:20.753151 7074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:20.753171 7071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:20.753144 7072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:20.753902 7066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:20.754987 7066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:20.756158 7066 hybrid_clock.cc:648] HybridClock initialized: now 1790422820756138 us; error 32 us; skew 500 ppm
Sep 26 11:40:20 dist-test-slave-xxxx krb5kdc[7053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422820, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:20.758713 7066 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:40:20.759605 7066 webserver.cc:460] Webserver started at http://127.6.96.62:42475/ using document root <none> and password file <none>
I20260926 11:40:20.760126 7066 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:20.760200 7066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:20.760389 7066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:20.762349 7066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "ec7781aacd0e469cbe4fa2371a26a1ee"
format_stamp: "Formatted at 2026-09-26 11:40:20 on dist-test-slave-xxxx"
server_key: "d87ac48cd4abea4b91e564927c6ff39b"
server_key_iv: "bffadb900602877972ce35dce5036e9f"
server_key_version: "encryptionkey@0"
I20260926 11:40:20.762944 7066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "ec7781aacd0e469cbe4fa2371a26a1ee"
format_stamp: "Formatted at 2026-09-26 11:40:20 on dist-test-slave-xxxx"
server_key: "d87ac48cd4abea4b91e564927c6ff39b"
server_key_iv: "bffadb900602877972ce35dce5036e9f"
server_key_version: "encryptionkey@0"
I20260926 11:40:20.766433 7066 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260926 11:40:20.768821 7081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:20.769948 7066 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:20.770053 7066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "ec7781aacd0e469cbe4fa2371a26a1ee"
format_stamp: "Formatted at 2026-09-26 11:40:20 on dist-test-slave-xxxx"
server_key: "d87ac48cd4abea4b91e564927c6ff39b"
server_key_iv: "bffadb900602877972ce35dce5036e9f"
server_key_version: "encryptionkey@0"
I20260926 11:40:20.770143 7066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:20.790234 7066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:20.793356 7066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:20.793540 7066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:20.800207 7066 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:41281
I20260926 11:40:20.800216 7133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:41281 every 8 connection(s)
I20260926 11:40:20.801117 7066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:40:20.803934 7134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:20.809309 7134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec7781aacd0e469cbe4fa2371a26a1ee: Bootstrap starting.
I20260926 11:40:20.810119 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 7066
I20260926 11:40:20.810227 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:40:20.810487 6528 external_mini_cluster.cc:1428] Setting key f250eea6fe81c061bbcf4eb85645d9b1
I20260926 11:40:20.812075 7134 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec7781aacd0e469cbe4fa2371a26a1ee: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:20.812788 7134 log.cc:824] T 00000000000000000000000000000000 P ec7781aacd0e469cbe4fa2371a26a1ee: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:20.814939 7134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec7781aacd0e469cbe4fa2371a26a1ee: No bootstrap required, opened a new log
I20260926 11:40:20.819100 7134 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec7781aacd0e469cbe4fa2371a26a1ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec7781aacd0e469cbe4fa2371a26a1ee" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 41281 } }
I20260926 11:40:20.819322 7134 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec7781aacd0e469cbe4fa2371a26a1ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Sep 26 11:40:20 dist-test-slave-xxxx krb5kdc[7053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:20.819357 7134 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec7781aacd0e469cbe4fa2371a26a1ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec7781aacd0e469cbe4fa2371a26a1ee, State: Initialized, Role: FOLLOWER
I20260926 11:40:20.819813 7134 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec7781aacd0e469cbe4fa2371a26a1ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec7781aacd0e469cbe4fa2371a26a1ee" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 41281 } }
I20260926 11:40:20.819938 7134 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec7781aacd0e469cbe4fa2371a26a1ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:40:20.819998 7134 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec7781aacd0e469cbe4fa2371a26a1ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:40:20.820091 7134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec7781aacd0e469cbe4fa2371a26a1ee [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:20.821098 7134 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec7781aacd0e469cbe4fa2371a26a1ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec7781aacd0e469cbe4fa2371a26a1ee" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 41281 } }
I20260926 11:40:20.821422 7134 leader_election.cc:302] T 00000000000000000000000000000000 P ec7781aacd0e469cbe4fa2371a26a1ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec7781aacd0e469cbe4fa2371a26a1ee; no voters:
I20260926 11:40:20.821708 7134 leader_election.cc:288] T 00000000000000000000000000000000 P ec7781aacd0e469cbe4fa2371a26a1ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:40:20.821821 7139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec7781aacd0e469cbe4fa2371a26a1ee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:20.822026 7139 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec7781aacd0e469cbe4fa2371a26a1ee [term 1 LEADER]: Becoming Leader. State: Replica: ec7781aacd0e469cbe4fa2371a26a1ee, State: Running, Role: LEADER
I20260926 11:40:20.822371 7139 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec7781aacd0e469cbe4fa2371a26a1ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec7781aacd0e469cbe4fa2371a26a1ee" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 41281 } }
I20260926 11:40:20.823082 7134 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec7781aacd0e469cbe4fa2371a26a1ee [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:40:20.824055 7140 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec7781aacd0e469cbe4fa2371a26a1ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec7781aacd0e469cbe4fa2371a26a1ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec7781aacd0e469cbe4fa2371a26a1ee" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 41281 } } }
I20260926 11:40:20.824177 7140 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec7781aacd0e469cbe4fa2371a26a1ee [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:20.824553 7141 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec7781aacd0e469cbe4fa2371a26a1ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec7781aacd0e469cbe4fa2371a26a1ee. Latest consensus state: current_term: 1 leader_uuid: "ec7781aacd0e469cbe4fa2371a26a1ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec7781aacd0e469cbe4fa2371a26a1ee" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 41281 } } }
I20260926 11:40:20.824692 7141 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec7781aacd0e469cbe4fa2371a26a1ee [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:20.824987 7148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:40:20.825129 7137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:20.811639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37720 (local address 127.6.96.62:41281)
0926 11:40:20.812103 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:20.812120 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:20.812166 (+ 46us) server_negotiation.cc:407] Connection header received
0926 11:40:20.812795 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:20.812811 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:20.813213 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:20.813576 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:20.814391 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:20.815426 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:20.816556 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:20.817045 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:20.817156 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:20.819911 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:20.819928 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:20.819946 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:20.819980 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:20.821686 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:20.822260 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:20.822265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:20.822271 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:20.822402 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:20.822699 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:20.822703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:20.822704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:20.823265 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:20.823464 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:20.823772 (+ 308us) server_negotiation.cc:299] Negotiation successful
0926 11:40:20.824113 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":162,"threads_started":1}
I20260926 11:40:20.828498 7148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:40:20.834267 7148 catalog_manager.cc:1347] Generated new cluster ID: b9d536b1e85e43c2850762d6dd6e8e33
I20260926 11:40:20.834343 7148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:40:20.844379 7148 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:40:20.845703 7148 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:40:20.855002 7148 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec7781aacd0e469cbe4fa2371a26a1ee: Generated new TSK 0
I20260926 11:40:20.855811 7148 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.96.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
I20260926 11:40:20.886085 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:41281
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:40061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:20.993207 7160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:20.993458 7160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:20.993517 7160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:20.997475 7160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:20.997534 7160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:20.997620 7160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:40:21.002735 7160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:40061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:41281
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:21.003762 7160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:21.004769 7160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 11:40:21.013988 7160 server_base.cc:1034] running on GCE node
W20260926 11:40:21.013952 7165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:21.013967 7166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:21.013952 7168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:21.014590 7160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:21.015223 7160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:21.016369 7160 hybrid_clock.cc:648] HybridClock initialized: now 1790422821016351 us; error 30 us; skew 500 ppm
Sep 26 11:40:21 dist-test-slave-xxxx krb5kdc[7053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422821, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:21.018909 7160 init.cc:377] Logged in from keytab as kudu/127.6.96.1@KRBTEST.COM (short username kudu)
I20260926 11:40:21.019691 7160 webserver.cc:460] Webserver started at http://127.6.96.1:41279/ using document root <none> and password file <none>
I20260926 11:40:21.020184 7160 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:21.020246 7160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:21.020402 7160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:21.022392 7160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "ac7bde4bdbe64e6781cadc67f4395fe9"
format_stamp: "Formatted at 2026-09-26 11:40:21 on dist-test-slave-xxxx"
server_key: "b91f8c399e8428206e910c3093d71adb"
server_key_iv: "bb92485fc7b39bcd042294e936e7aea3"
server_key_version: "encryptionkey@0"
I20260926 11:40:21.022961 7160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "ac7bde4bdbe64e6781cadc67f4395fe9"
format_stamp: "Formatted at 2026-09-26 11:40:21 on dist-test-slave-xxxx"
server_key: "b91f8c399e8428206e910c3093d71adb"
server_key_iv: "bb92485fc7b39bcd042294e936e7aea3"
server_key_version: "encryptionkey@0"
I20260926 11:40:21.026526 7160 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:40:21.028914 7175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:21.030145 7160 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:21.030247 7160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "ac7bde4bdbe64e6781cadc67f4395fe9"
format_stamp: "Formatted at 2026-09-26 11:40:21 on dist-test-slave-xxxx"
server_key: "b91f8c399e8428206e910c3093d71adb"
server_key_iv: "bb92485fc7b39bcd042294e936e7aea3"
server_key_version: "encryptionkey@0"
I20260926 11:40:21.030332 7160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:21.042202 7160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:21.044996 7160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:21.045169 7160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:21.045706 7160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:21.046454 7160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:21.046500 7160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:21.046557 7160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:21.046583 7160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:21.054629 7160 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:42689
I20260926 11:40:21.054649 7288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:42689 every 8 connection(s)
I20260926 11:40:21.055503 7160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:40:21.062096 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 7160
I20260926 11:40:21.062191 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:40:21.062405 6528 external_mini_cluster.cc:1428] Setting key 9335a613b4ae020a44bb261ab9fd30f1
Sep 26 11:40:21 dist-test-slave-xxxx krb5kdc[7053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422821, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:21.068580 7137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:21.057240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:45613 (local address 127.6.96.62:41281)
0926 11:40:21.057364 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:21.057368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:21.058174 (+ 806us) server_negotiation.cc:407] Connection header received
0926 11:40:21.059470 (+ 1296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:21.059473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:21.059526 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:21.059612 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:21.060720 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.061237 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.062345 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.062606 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.062693 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:21.065136 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:21.065139 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:21.065141 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:21.065163 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:21.066587 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:21.067068 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:21.067072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:21.067073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:21.067158 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:21.067484 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:21.067488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:21.067489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:21.067715 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:21.067808 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:21.068280 (+ 472us) server_negotiation.cc:299] Negotiation successful
0926 11:40:21.068402 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 11:40:21.069288 7291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:21.057608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41281 (local address 127.6.96.1:45613)
0926 11:40:21.058019 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:21.058059 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:21.059272 (+ 1213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:21.059784 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:21.059791 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:21.060130 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:21.060582 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:21.060594 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.061345 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.061348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:21.062234 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:21.062244 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.062737 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.062741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:21.062929 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:21.063572 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:21.063592 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:21.064969 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:21.066698 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:21.066703 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:21.066713 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:21.066985 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:21.067238 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:21.067240 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:21.067242 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:21.067393 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:21.067796 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:21.067801 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:21.068016 (+ 215us) client_negotiation.cc:769] Sending connection context
0926 11:40:21.068190 (+ 174us) client_negotiation.cc:240] Negotiation successful
0926 11:40:21.068425 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":103,"threads_started":1}
I20260926 11:40:21.070371 7289 heartbeater.cc:347] Connected to a master server at 127.6.96.62:41281
I20260926 11:40:21.070598 7289 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.6.96.2@KRBTEST.COM; defaulting to no policy
I20260926 11:40:21.071105 7289 heartbeater.cc:511] Master 127.6.96.62:41281 requested a full tablet report, sending...
I20260926 11:40:21.072664 7098 ts_manager.cc:194] Registered new tserver with Master: ac7bde4bdbe64e6781cadc67f4395fe9 (127.6.96.1:42689)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.2@KRBTEST.COM" created.
I20260926 11:40:21.074107 7098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.1@KRBTEST.COM'} at 127.6.96.1:45613
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
I20260926 11:40:21.089318 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:41281
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:40061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:21.196269 7294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:21.196511 7294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:21.196554 7294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:21.200515 7294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:21.200574 7294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:21.200637 7294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:40:21.205741 7294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:40061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:41281
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:21.206760 7294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:21.207779 7294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:21.217345 7299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:21.217309 7300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:21.217310 7302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:21.217724 7294 server_base.cc:1034] running on GCE node
I20260926 11:40:21.218125 7294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:21.218780 7294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:21.219991 7294 hybrid_clock.cc:648] HybridClock initialized: now 1790422821219937 us; error 33 us; skew 500 ppm
Sep 26 11:40:21 dist-test-slave-xxxx krb5kdc[7053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422821, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:21.222687 7294 init.cc:377] Logged in from keytab as kudu/127.6.96.2@KRBTEST.COM (short username kudu)
I20260926 11:40:21.223759 7294 webserver.cc:460] Webserver started at http://127.6.96.2:45631/ using document root <none> and password file <none>
I20260926 11:40:21.224457 7294 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:21.224525 7294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:21.224710 7294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:21.226759 7294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "1c9de3a72595443e934b522024c66ae0"
format_stamp: "Formatted at 2026-09-26 11:40:21 on dist-test-slave-xxxx"
server_key: "d041814a03464cc18123c91b9dbd6379"
server_key_iv: "ab6db9077ea321e5f0d8e4b2bd6c113f"
server_key_version: "encryptionkey@0"
I20260926 11:40:21.227319 7294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "1c9de3a72595443e934b522024c66ae0"
format_stamp: "Formatted at 2026-09-26 11:40:21 on dist-test-slave-xxxx"
server_key: "d041814a03464cc18123c91b9dbd6379"
server_key_iv: "ab6db9077ea321e5f0d8e4b2bd6c113f"
server_key_version: "encryptionkey@0"
I20260926 11:40:21.230952 7294 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:40:21.233268 7309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:21.234591 7294 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:21.234709 7294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "1c9de3a72595443e934b522024c66ae0"
format_stamp: "Formatted at 2026-09-26 11:40:21 on dist-test-slave-xxxx"
server_key: "d041814a03464cc18123c91b9dbd6379"
server_key_iv: "ab6db9077ea321e5f0d8e4b2bd6c113f"
server_key_version: "encryptionkey@0"
I20260926 11:40:21.234800 7294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:21.248313 7294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:21.251187 7294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:21.251366 7294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:21.251935 7294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:21.252770 7294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:21.252817 7294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:21.252871 7294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:21.252898 7294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:21.260923 7294 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:45717
I20260926 11:40:21.260946 7422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:45717 every 8 connection(s)
I20260926 11:40:21.261807 7294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:40:21.264884 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 7294
I20260926 11:40:21.264983 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:40:21.265237 6528 external_mini_cluster.cc:1428] Setting key fa6bab60296c66ebab09e331b7974953
Sep 26 11:40:21 dist-test-slave-xxxx krb5kdc[7053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422821, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.96.3@KRBTEST.COM; defaulting to no policy
I20260926 11:40:21.275837 7137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:21.263546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:53965 (local address 127.6.96.62:41281)
0926 11:40:21.263674 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:21.263677 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:21.264451 (+ 774us) server_negotiation.cc:407] Connection header received
0926 11:40:21.265795 (+ 1344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:21.265798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:21.265849 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:21.265933 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:21.267294 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.268015 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.269593 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.269808 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.269882 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:21.272600 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:21.272603 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:21.272606 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:21.272629 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:21.273948 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:21.274508 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:21.274511 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:21.274512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:21.274598 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:21.274921 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:21.274923 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:21.274924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:21.275089 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:21.275163 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:21.275625 (+ 462us) server_negotiation.cc:299] Negotiation successful
0926 11:40:21.275721 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 11:40:21.276618 7425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:21.263869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41281 (local address 127.6.96.2:53965)
0926 11:40:21.264293 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:21.264335 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:21.265588 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:21.266130 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:21.266139 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:21.266605 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:21.267158 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:21.267172 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.268152 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.268156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:21.269484 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:21.269494 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.269941 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.269944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:21.270136 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:21.270715 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:21.270736 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:21.272432 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:21.274050 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:21.274056 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:21.274072 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:21.274414 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:21.274689 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:21.274692 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:21.274693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:21.274818 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:21.275163 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:21.275167 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:21.275408 (+ 241us) client_negotiation.cc:769] Sending connection context
0926 11:40:21.275558 (+ 150us) client_negotiation.cc:240] Negotiation successful
0926 11:40:21.275801 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":97,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.3@KRBTEST.COM" created.
I20260926 11:40:21.277745 7423 heartbeater.cc:347] Connected to a master server at 127.6.96.62:41281
I20260926 11:40:21.277971 7423 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:21.278508 7423 heartbeater.cc:511] Master 127.6.96.62:41281 requested a full tablet report, sending...
I20260926 11:40:21.279546 7098 ts_manager.cc:194] Registered new tserver with Master: 1c9de3a72595443e934b522024c66ae0 (127.6.96.2:45717)
I20260926 11:40:21.280102 7098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.2@KRBTEST.COM'} at 127.6.96.2:53965
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
I20260926 11:40:21.291141 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:41281
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:40061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:21.397899 7428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:21.398213 7428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:21.398277 7428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:21.402251 7428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:21.402304 7428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:21.402383 7428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:40:21.407531 7428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:40061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:41281
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:21.408519 7428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:21.409492 7428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:21.418972 7434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:21.419041 7428 server_base.cc:1034] running on GCE node
W20260926 11:40:21.418972 7433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:21.418975 7436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:21.419566 7428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:21.420186 7428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:21.421352 7428 hybrid_clock.cc:648] HybridClock initialized: now 1790422821421326 us; error 37 us; skew 500 ppm
Sep 26 11:40:21 dist-test-slave-xxxx krb5kdc[7053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422821, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:21.430367 7428 init.cc:377] Logged in from keytab as kudu/127.6.96.3@KRBTEST.COM (short username kudu)
I20260926 11:40:21.431308 7428 webserver.cc:460] Webserver started at http://127.6.96.3:42241/ using document root <none> and password file <none>
I20260926 11:40:21.431843 7428 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:21.431907 7428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:21.432099 7428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:21.434180 7428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "f195cf07e87343d184f7b4e02aaaa174"
format_stamp: "Formatted at 2026-09-26 11:40:21 on dist-test-slave-xxxx"
server_key: "653ceea92601dc59d2b36fd1f81a4d24"
server_key_iv: "78deedf1d9e5924c6d1d4cf285937620"
server_key_version: "encryptionkey@0"
I20260926 11:40:21.434719 7428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "f195cf07e87343d184f7b4e02aaaa174"
format_stamp: "Formatted at 2026-09-26 11:40:21 on dist-test-slave-xxxx"
server_key: "653ceea92601dc59d2b36fd1f81a4d24"
server_key_iv: "78deedf1d9e5924c6d1d4cf285937620"
server_key_version: "encryptionkey@0"
I20260926 11:40:21.438344 7428 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 11:40:21.440704 7443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:21.441854 7428 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:21.441955 7428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "f195cf07e87343d184f7b4e02aaaa174"
format_stamp: "Formatted at 2026-09-26 11:40:21 on dist-test-slave-xxxx"
server_key: "653ceea92601dc59d2b36fd1f81a4d24"
server_key_iv: "78deedf1d9e5924c6d1d4cf285937620"
server_key_version: "encryptionkey@0"
I20260926 11:40:21.442049 7428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:21.467470 7428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:21.470256 7428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:21.470418 7428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:21.471020 7428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:21.471822 7428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:21.471866 7428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:21.471907 7428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:21.471935 7428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:21.480175 7428 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:46869
I20260926 11:40:21.480201 7556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:46869 every 8 connection(s)
I20260926 11:40:21.481022 7428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:40:21.486728 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 7428
I20260926 11:40:21.486819 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:40:21.487064 6528 external_mini_cluster.cc:1428] Setting key 4f16c4830c2bf673f89945fbd230670e
Sep 26 11:40:21 dist-test-slave-xxxx krb5kdc[7053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422821, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:21.493988 7137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:21.482706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:40619 (local address 127.6.96.62:41281)
0926 11:40:21.482854 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:21.482859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:21.483693 (+ 834us) server_negotiation.cc:407] Connection header received
0926 11:40:21.485037 (+ 1344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:21.485041 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:21.485099 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:21.485175 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:21.486320 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.486916 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.488327 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.488563 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.488650 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:21.491045 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:21.491048 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:21.491050 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:21.491074 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:21.492163 (+ 1089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:21.492625 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:21.492629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:21.492631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:21.492703 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:21.493002 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:21.493005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:21.493007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:21.493172 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:21.493254 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:21.493723 (+ 469us) server_negotiation.cc:299] Negotiation successful
0926 11:40:21.493835 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 11:40:21.494756 7559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:21.483106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41281 (local address 127.6.96.3:40619)
0926 11:40:21.483546 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:21.483589 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:21.484796 (+ 1207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:21.485356 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:21.485362 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:21.485708 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:21.486179 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:21.486193 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.487032 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.487034 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:21.488216 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:21.488226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.488697 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.488701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:21.488891 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:21.489418 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:21.489434 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:21.490822 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:21.492270 (+ 1448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:21.492276 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:21.492289 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:21.492531 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:21.492783 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:21.492786 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:21.492788 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:21.492912 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:21.493264 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:21.493270 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:21.493502 (+ 232us) client_negotiation.cc:769] Sending connection context
0926 11:40:21.493668 (+ 166us) client_negotiation.cc:240] Negotiation successful
0926 11:40:21.493983 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":94,"threads_started":1}
I20260926 11:40:21.495761 7557 heartbeater.cc:347] Connected to a master server at 127.6.96.62:41281
I20260926 11:40:21.495971 7557 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:21.496407 7557 heartbeater.cc:511] Master 127.6.96.62:41281 requested a full tablet report, sending...
I20260926 11:40:21.497283 7098 ts_manager.cc:194] Registered new tserver with Master: f195cf07e87343d184f7b4e02aaaa174 (127.6.96.3:46869)
I20260926 11:40:21.497795 7098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.3@KRBTEST.COM'} at 127.6.96.3:40619
I20260926 11:40:21.500417 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 11:40:21.509524 7137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:21.503431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37728 (local address 127.6.96.62:41281)
0926 11:40:21.503541 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:21.503545 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:21.503737 (+ 192us) server_negotiation.cc:407] Connection header received
0926 11:40:21.503838 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:21.503841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:21.503883 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:21.503938 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:21.504492 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.504984 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.505732 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.505924 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.505985 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:21.507083 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:21.507087 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:21.507089 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:21.507109 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:21.508321 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:21.508635 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:21.508637 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:21.508639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:21.508701 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:21.508871 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:21.508873 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:21.508874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:21.509067 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:21.509160 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:21.509302 (+ 142us) server_negotiation.cc:299] Negotiation successful
0926 11:40:21.509382 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 11:40:21.513533 7098 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 11:40:21.516443 7098 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:40:21.529510 7568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:21.524104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:45717 (local address 127.0.0.1:47280)
0926 11:40:21.524474 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:21.524522 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:21.524671 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:21.525399 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:21.525404 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:21.525425 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:21.525553 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:21.525560 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.526994 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.526997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:21.528160 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:21.528169 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.529125 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.529127 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:21.529208 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:21.529238 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 11:40:21.529314 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 11:40:21.529399 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":1,"thread_start_us":84,"threads_started":1}
I20260926 11:40:21.529769 7570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:21.524238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47280 (local address 127.6.96.2:45717)
0926 11:40:21.524643 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:21.524655 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:21.524678 (+ 23us) server_negotiation.cc:407] Connection header received
0926 11:40:21.524860 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:21.524866 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:21.525073 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:21.525222 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:21.525699 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.526872 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.528291 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.529016 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.529130 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:21.529271 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:21.529352 (+ 81us) server_negotiation.cc:299] Negotiation successful
0926 11:40:21.529438 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":114,"threads_started":1}
I20260926 11:40:21.530135 7571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:21.524794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:46869 (local address 127.0.0.1:52366)
0926 11:40:21.525549 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:21.525561 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:21.525655 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:21.526243 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:21.526246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:21.526261 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:21.526359 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:21.526365 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.527483 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.527487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:21.528657 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:21.528667 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.529803 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.529806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:21.529893 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:21.529916 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 11:40:21.529980 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 11:40:21.530033 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"mutex_wait_us":52,"thread_start_us":79,"threads_started":1}
I20260926 11:40:21.530478 7573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:21.525170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52366 (local address 127.6.96.3:46869)
0926 11:40:21.525912 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:21.525920 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:21.525938 (+ 18us) server_negotiation.cc:407] Connection header received
0926 11:40:21.525991 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:21.525995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:21.526123 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:21.526240 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:21.526446 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.527348 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.528778 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.529538 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.529648 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:21.529776 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:21.530283 (+ 507us) server_negotiation.cc:299] Negotiation successful
0926 11:40:21.530368 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":96,"threads_started":1}
I20260926 11:40:21.531126 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:21.524125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:42689 (local address 127.0.0.1:52618)
0926 11:40:21.525037 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:21.525049 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:21.525134 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:21.525722 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:21.525725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:21.525743 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:21.525825 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:21.525831 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.527239 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.527242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:21.528657 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:21.528667 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.530723 (+ 2056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.530727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:21.530822 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:21.530866 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 11:40:21.530936 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 11:40:21.530999 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"mutex_wait_us":45,"thread_start_us":80,"threads_started":1}
I20260926 11:40:21.531620 7572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:21.524901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52618 (local address 127.6.96.1:42689)
0926 11:40:21.525298 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:21.525309 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:21.525329 (+ 20us) server_negotiation.cc:407] Connection header received
0926 11:40:21.525398 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:21.525402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:21.525574 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:21.525963 (+ 389us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:21.526114 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.527125 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.529839 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.530595 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.530697 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:21.530850 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:21.531373 (+ 523us) server_negotiation.cc:299] Negotiation successful
0926 11:40:21.531469 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":102,"threads_started":1}
I20260926 11:40:21.532713 7357 tablet_service.cc:1467] Processing CreateTablet for tablet e5cfd40ded224a6faddcb3234ad389b3 (DEFAULT_TABLE table=test-table [id=ef9c936a08bf43e4a6e36e9e47bb0bd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:21.533032 7491 tablet_service.cc:1467] Processing CreateTablet for tablet e5cfd40ded224a6faddcb3234ad389b3 (DEFAULT_TABLE table=test-table [id=ef9c936a08bf43e4a6e36e9e47bb0bd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:21.533629 7357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5cfd40ded224a6faddcb3234ad389b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:21.534029 7491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5cfd40ded224a6faddcb3234ad389b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:21.534051 7223 tablet_service.cc:1467] Processing CreateTablet for tablet e5cfd40ded224a6faddcb3234ad389b3 (DEFAULT_TABLE table=test-table [id=ef9c936a08bf43e4a6e36e9e47bb0bd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:21.535014 7223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5cfd40ded224a6faddcb3234ad389b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:21.540968 7575 tablet_bootstrap.cc:492] T e5cfd40ded224a6faddcb3234ad389b3 P f195cf07e87343d184f7b4e02aaaa174: Bootstrap starting.
I20260926 11:40:21.541821 7576 tablet_bootstrap.cc:492] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9: Bootstrap starting.
I20260926 11:40:21.542886 7575 tablet_bootstrap.cc:654] T e5cfd40ded224a6faddcb3234ad389b3 P f195cf07e87343d184f7b4e02aaaa174: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:21.543171 7574 tablet_bootstrap.cc:492] T e5cfd40ded224a6faddcb3234ad389b3 P 1c9de3a72595443e934b522024c66ae0: Bootstrap starting.
I20260926 11:40:21.543608 7576 tablet_bootstrap.cc:654] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:21.544029 7575 log.cc:824] T e5cfd40ded224a6faddcb3234ad389b3 P f195cf07e87343d184f7b4e02aaaa174: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:21.544813 7576 log.cc:824] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:21.545815 7574 tablet_bootstrap.cc:654] T e5cfd40ded224a6faddcb3234ad389b3 P 1c9de3a72595443e934b522024c66ae0: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:21.545888 7575 tablet_bootstrap.cc:492] T e5cfd40ded224a6faddcb3234ad389b3 P f195cf07e87343d184f7b4e02aaaa174: No bootstrap required, opened a new log
I20260926 11:40:21.546069 7575 ts_tablet_manager.cc:1397] T e5cfd40ded224a6faddcb3234ad389b3 P f195cf07e87343d184f7b4e02aaaa174: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260926 11:40:21.546495 7574 log.cc:824] T e5cfd40ded224a6faddcb3234ad389b3 P 1c9de3a72595443e934b522024c66ae0: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:21.548367 7574 tablet_bootstrap.cc:492] T e5cfd40ded224a6faddcb3234ad389b3 P 1c9de3a72595443e934b522024c66ae0: No bootstrap required, opened a new log
I20260926 11:40:21.548550 7574 ts_tablet_manager.cc:1397] T e5cfd40ded224a6faddcb3234ad389b3 P 1c9de3a72595443e934b522024c66ae0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 11:40:21.549628 7575 raft_consensus.cc:348] T e5cfd40ded224a6faddcb3234ad389b3 P f195cf07e87343d184f7b4e02aaaa174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c9de3a72595443e934b522024c66ae0" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 45717 } } peers { permanent_uuid: "ac7bde4bdbe64e6781cadc67f4395fe9" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42689 } } peers { permanent_uuid: "f195cf07e87343d184f7b4e02aaaa174" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 46869 } }
I20260926 11:40:21.549896 7575 raft_consensus.cc:374] T e5cfd40ded224a6faddcb3234ad389b3 P f195cf07e87343d184f7b4e02aaaa174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:21.549947 7575 raft_consensus.cc:729] T e5cfd40ded224a6faddcb3234ad389b3 P f195cf07e87343d184f7b4e02aaaa174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f195cf07e87343d184f7b4e02aaaa174, State: Initialized, Role: FOLLOWER
I20260926 11:40:21.550395 7575 consensus_queue.cc:260] T e5cfd40ded224a6faddcb3234ad389b3 P f195cf07e87343d184f7b4e02aaaa174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c9de3a72595443e934b522024c66ae0" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 45717 } } peers { permanent_uuid: "ac7bde4bdbe64e6781cadc67f4395fe9" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42689 } } peers { permanent_uuid: "f195cf07e87343d184f7b4e02aaaa174" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 46869 } }
I20260926 11:40:21.551085 7557 heartbeater.cc:503] Master 127.6.96.62:41281 was elected leader, sending a full tablet report...
I20260926 11:40:21.551998 7576 tablet_bootstrap.cc:492] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9: No bootstrap required, opened a new log
I20260926 11:40:21.552261 7576 ts_tablet_manager.cc:1397] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260926 11:40:21.552222 7574 raft_consensus.cc:348] T e5cfd40ded224a6faddcb3234ad389b3 P 1c9de3a72595443e934b522024c66ae0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c9de3a72595443e934b522024c66ae0" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 45717 } } peers { permanent_uuid: "ac7bde4bdbe64e6781cadc67f4395fe9" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42689 } } peers { permanent_uuid: "f195cf07e87343d184f7b4e02aaaa174" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 46869 } }
I20260926 11:40:21.552521 7574 raft_consensus.cc:374] T e5cfd40ded224a6faddcb3234ad389b3 P 1c9de3a72595443e934b522024c66ae0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:21.552599 7574 raft_consensus.cc:729] T e5cfd40ded224a6faddcb3234ad389b3 P 1c9de3a72595443e934b522024c66ae0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c9de3a72595443e934b522024c66ae0, State: Initialized, Role: FOLLOWER
I20260926 11:40:21.553028 7574 consensus_queue.cc:260] T e5cfd40ded224a6faddcb3234ad389b3 P 1c9de3a72595443e934b522024c66ae0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c9de3a72595443e934b522024c66ae0" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 45717 } } peers { permanent_uuid: "ac7bde4bdbe64e6781cadc67f4395fe9" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42689 } } peers { permanent_uuid: "f195cf07e87343d184f7b4e02aaaa174" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 46869 } }
I20260926 11:40:21.553689 7423 heartbeater.cc:503] Master 127.6.96.62:41281 was elected leader, sending a full tablet report...
I20260926 11:40:21.554056 7574 ts_tablet_manager.cc:1428] T e5cfd40ded224a6faddcb3234ad389b3 P 1c9de3a72595443e934b522024c66ae0: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 11:40:21.554738 7575 ts_tablet_manager.cc:1428] T e5cfd40ded224a6faddcb3234ad389b3 P f195cf07e87343d184f7b4e02aaaa174: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260926 11:40:21.555918 7576 raft_consensus.cc:348] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c9de3a72595443e934b522024c66ae0" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 45717 } } peers { permanent_uuid: "ac7bde4bdbe64e6781cadc67f4395fe9" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42689 } } peers { permanent_uuid: "f195cf07e87343d184f7b4e02aaaa174" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 46869 } }
I20260926 11:40:21.556180 7576 raft_consensus.cc:374] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:21.556243 7576 raft_consensus.cc:729] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac7bde4bdbe64e6781cadc67f4395fe9, State: Initialized, Role: FOLLOWER
I20260926 11:40:21.556653 7576 consensus_queue.cc:260] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c9de3a72595443e934b522024c66ae0" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 45717 } } peers { permanent_uuid: "ac7bde4bdbe64e6781cadc67f4395fe9" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42689 } } peers { permanent_uuid: "f195cf07e87343d184f7b4e02aaaa174" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 46869 } }
I20260926 11:40:21.557243 7289 heartbeater.cc:503] Master 127.6.96.62:41281 was elected leader, sending a full tablet report...
I20260926 11:40:21.557415 7576 ts_tablet_manager.cc:1428] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260926 11:40:21.721220 7582 raft_consensus.cc:482] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:40:21.721414 7582 raft_consensus.cc:504] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c9de3a72595443e934b522024c66ae0" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 45717 } } peers { permanent_uuid: "ac7bde4bdbe64e6781cadc67f4395fe9" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42689 } } peers { permanent_uuid: "f195cf07e87343d184f7b4e02aaaa174" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 46869 } }
I20260926 11:40:21.722489 7582 leader_election.cc:288] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c9de3a72595443e934b522024c66ae0 (127.6.96.2:45717), f195cf07e87343d184f7b4e02aaaa174 (127.6.96.3:46869)
I20260926 11:40:21.726613 7570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:21.722889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:44819 (local address 127.6.96.2:45717)
0926 11:40:21.722998 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:21.723001 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:21.723158 (+ 157us) server_negotiation.cc:407] Connection header received
0926 11:40:21.723329 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:21.723332 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:21.723402 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:21.723491 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:21.723781 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.724487 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.725667 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.726092 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.726180 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:21.726241 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:21.726422 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 11:40:21.726490 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 11:40:21.726637 7584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:21.723213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:46869 (local address 127.6.96.1:35457)
0926 11:40:21.723567 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:21.723580 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:21.723679 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:21.723871 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:21.723874 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:21.723888 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:21.723989 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:21.723995 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.724773 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.724776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:21.725614 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:21.725625 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.726236 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.726239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:21.726343 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:21.726360 (+ 17us) client_negotiation.cc:769] Sending connection context
0926 11:40:21.726428 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 11:40:21.726489 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":108,"threads_started":1}
I20260926 11:40:21.726630 7573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:21.723351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:35457 (local address 127.6.96.3:46869)
0926 11:40:21.723491 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:21.723495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:21.723667 (+ 172us) server_negotiation.cc:407] Connection header received
0926 11:40:21.723749 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:21.723752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:21.723795 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:21.723847 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:21.724112 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.724680 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.725728 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.726124 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.726217 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:21.726275 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:21.726456 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 11:40:21.726521 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 11:40:21.726647 7583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:21.722748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:45717 (local address 127.6.96.1:44819)
0926 11:40:21.723067 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:21.723090 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:21.723203 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:21.723526 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:21.723529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:21.723550 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:21.723665 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:21.723672 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.724595 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.724599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:21.725549 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:21.725558 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.726212 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.726215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:21.726305 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:21.726332 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 11:40:21.726390 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 11:40:21.726452 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":147,"threads_started":1}
I20260926 11:40:21.727273 7511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5cfd40ded224a6faddcb3234ad389b3" candidate_uuid: "ac7bde4bdbe64e6781cadc67f4395fe9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f195cf07e87343d184f7b4e02aaaa174" is_pre_election: true
I20260926 11:40:21.727595 7511 raft_consensus.cc:2454] T e5cfd40ded224a6faddcb3234ad389b3 P f195cf07e87343d184f7b4e02aaaa174 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac7bde4bdbe64e6781cadc67f4395fe9 in term 0.
I20260926 11:40:21.727634 7377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5cfd40ded224a6faddcb3234ad389b3" candidate_uuid: "ac7bde4bdbe64e6781cadc67f4395fe9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c9de3a72595443e934b522024c66ae0" is_pre_election: true
I20260926 11:40:21.727900 7377 raft_consensus.cc:2454] T e5cfd40ded224a6faddcb3234ad389b3 P 1c9de3a72595443e934b522024c66ae0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac7bde4bdbe64e6781cadc67f4395fe9 in term 0.
I20260926 11:40:21.728061 7176 leader_election.cc:302] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac7bde4bdbe64e6781cadc67f4395fe9, f195cf07e87343d184f7b4e02aaaa174; no voters:
I20260926 11:40:21.728344 7582 raft_consensus.cc:2781] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:40:21.728421 7582 raft_consensus.cc:482] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:40:21.728449 7582 raft_consensus.cc:3037] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:21.729460 7582 raft_consensus.cc:504] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c9de3a72595443e934b522024c66ae0" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 45717 } } peers { permanent_uuid: "ac7bde4bdbe64e6781cadc67f4395fe9" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42689 } } peers { permanent_uuid: "f195cf07e87343d184f7b4e02aaaa174" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 46869 } }
I20260926 11:40:21.729777 7582 leader_election.cc:288] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9 [CANDIDATE]: Term 1 election: Requested vote from peers 1c9de3a72595443e934b522024c66ae0 (127.6.96.2:45717), f195cf07e87343d184f7b4e02aaaa174 (127.6.96.3:46869)
I20260926 11:40:21.730103 7377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5cfd40ded224a6faddcb3234ad389b3" candidate_uuid: "ac7bde4bdbe64e6781cadc67f4395fe9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c9de3a72595443e934b522024c66ae0"
I20260926 11:40:21.730113 7511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5cfd40ded224a6faddcb3234ad389b3" candidate_uuid: "ac7bde4bdbe64e6781cadc67f4395fe9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f195cf07e87343d184f7b4e02aaaa174"
I20260926 11:40:21.730223 7511 raft_consensus.cc:3037] T e5cfd40ded224a6faddcb3234ad389b3 P f195cf07e87343d184f7b4e02aaaa174 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:21.730223 7377 raft_consensus.cc:3037] T e5cfd40ded224a6faddcb3234ad389b3 P 1c9de3a72595443e934b522024c66ae0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:21.731509 7377 raft_consensus.cc:2454] T e5cfd40ded224a6faddcb3234ad389b3 P 1c9de3a72595443e934b522024c66ae0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac7bde4bdbe64e6781cadc67f4395fe9 in term 1.
I20260926 11:40:21.731534 7511 raft_consensus.cc:2454] T e5cfd40ded224a6faddcb3234ad389b3 P f195cf07e87343d184f7b4e02aaaa174 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac7bde4bdbe64e6781cadc67f4395fe9 in term 1.
I20260926 11:40:21.731820 7176 leader_election.cc:302] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c9de3a72595443e934b522024c66ae0, ac7bde4bdbe64e6781cadc67f4395fe9; no voters:
W20260926 11:40:21.731875 7558 tablet.cc:2369] T e5cfd40ded224a6faddcb3234ad389b3 P f195cf07e87343d184f7b4e02aaaa174: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:40:21.732084 7582 raft_consensus.cc:2781] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:21.732311 7582 raft_consensus.cc:686] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9 [term 1 LEADER]: Becoming Leader. State: Replica: ac7bde4bdbe64e6781cadc67f4395fe9, State: Running, Role: LEADER
I20260926 11:40:21.732626 7582 consensus_queue.cc:237] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c9de3a72595443e934b522024c66ae0" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 45717 } } peers { permanent_uuid: "ac7bde4bdbe64e6781cadc67f4395fe9" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42689 } } peers { permanent_uuid: "f195cf07e87343d184f7b4e02aaaa174" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 46869 } }
I20260926 11:40:21.735877 7097 catalog_manager.cc:5495] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac7bde4bdbe64e6781cadc67f4395fe9 (127.6.96.1). New cstate: current_term: 1 leader_uuid: "ac7bde4bdbe64e6781cadc67f4395fe9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c9de3a72595443e934b522024c66ae0" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 45717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac7bde4bdbe64e6781cadc67f4395fe9" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f195cf07e87343d184f7b4e02aaaa174" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 46869 } health_report { overall_health: UNKNOWN } } }
Sep 26 11:40:21 dist-test-slave-xxxx krb5kdc[7053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.1@KRBTEST.COM
W20260926 11:40:21.763034 7424 tablet.cc:2369] T e5cfd40ded224a6faddcb3234ad389b3 P 1c9de3a72595443e934b522024c66ae0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:40:21.764712 7572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:21.756680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52628 (local address 127.6.96.1:42689)
0926 11:40:21.756795 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:21.756799 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:21.757017 (+ 218us) server_negotiation.cc:407] Connection header received
0926 11:40:21.757104 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:21.757108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:21.757157 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:21.757226 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:21.757818 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.758308 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.759117 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.759330 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.759403 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:21.761611 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:21.761624 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:21.761638 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:21.761673 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:21.763072 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:21.763475 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:21.763481 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:21.763482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:21.763560 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:21.763800 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:21.763803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:21.763805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:21.764052 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:21.764247 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:21.764441 (+ 194us) server_negotiation.cc:299] Negotiation successful
0926 11:40:21.764554 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Sep 26 11:40:21 dist-test-slave-xxxx krb5kdc[7053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422821, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 11:40:21 dist-test-slave-xxxx krb5kdc[7053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422821, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.96.1@KRBTEST.COM
I20260926 11:40:21.789273 7572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:21.781419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52638 (local address 127.6.96.1:42689)
0926 11:40:21.781568 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:21.781572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:21.781700 (+ 128us) server_negotiation.cc:407] Connection header received
0926 11:40:21.781838 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:21.781841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:21.781882 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:21.781948 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:21.782616 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.783176 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.784048 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.784298 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.784385 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:21.786673 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:21.786677 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:21.786679 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:21.786703 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:21.787903 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:21.788326 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:21.788328 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:21.788329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:21.788398 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:21.788585 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:21.788587 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:21.788588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:21.788744 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:21.788853 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:21.789062 (+ 209us) server_negotiation.cc:299] Negotiation successful
0926 11:40:21.789154 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Sep 26 11:40:21 dist-test-slave-xxxx krb5kdc[7053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260926 11:40:21.806720 7290 tablet.cc:2369] T e5cfd40ded224a6faddcb3234ad389b3 P ac7bde4bdbe64e6781cadc67f4395fe9: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 11:40:21 dist-test-slave-xxxx krb5kdc[7053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.1@KRBTEST.COM
I20260926 11:40:21.814020 7572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:21.805271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52652 (local address 127.6.96.1:42689)
0926 11:40:21.805411 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:21.805417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:21.805560 (+ 143us) server_negotiation.cc:407] Connection header received
0926 11:40:21.805683 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:21.805685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:21.805729 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:21.805798 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:21.806550 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.807191 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.808378 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:21.808598 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:21.808674 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:21.811290 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:21.811294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:21.811296 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:21.811320 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:21.812559 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:21.812936 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:21.812939 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:21.812941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:21.813013 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:21.813281 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:21.813284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:21.813286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:21.813444 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:21.813572 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:21.813775 (+ 203us) server_negotiation.cc:299] Negotiation successful
0926 11:40:21.813878 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260926 11:40:21.815373 7203 tablet_service.cc:2908] Rejecting scan request for tablet e5cfd40ded224a6faddcb3234ad389b3: Uninitialized: clean time has not yet been initialized
I20260926 11:40:21.817214 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 7160
I20260926 11:40:21.822645 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 7294
I20260926 11:40:21.827082 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 7428
I20260926 11:40:21.831445 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 7066
2026-09-26T11:40:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2268 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:40:21 dist-test-slave-xxxx krb5kdc[7599](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:40:21 dist-test-slave-xxxx krb5kdc[7599](info): set up 2 sockets
Sep 26 11:40:21 dist-test-slave-xxxx krb5kdc[7599](info): commencing operation
krb5kdc: starting...
W20260926 11:40:22.778329 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:40:22 dist-test-slave-xxxx krb5kdc[7599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:40:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:40:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:40:22.897787 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:36643
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:46851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:36643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:23.003542 7612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:23.003799 7612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:23.003836 7612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:23.007809 7612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:40:23.007869 7612 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 11:40:23.007885 7612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:23.007902 7612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:23.007917 7612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:40:23.007933 7612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:40:23.013302 7612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:46851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:36643
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:36643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:23.014317 7612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:23.015329 7612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:23.023275 7620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:23.023298 7617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:23.023320 7618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:23.023564 7612 server_base.cc:1034] running on GCE node
I20260926 11:40:23.024154 7612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:23.025182 7612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:23.026340 7612 hybrid_clock.cc:648] HybridClock initialized: now 1790422823026317 us; error 36 us; skew 500 ppm
Sep 26 11:40:23 dist-test-slave-xxxx krb5kdc[7599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422823, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:23.028970 7612 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:40:23.029857 7612 webserver.cc:460] Webserver started at http://127.6.96.62:44911/ using document root <none> and password file <none>
I20260926 11:40:23.030383 7612 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:23.030447 7612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:23.030638 7612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:23.032680 7612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "900e2cd538cf416c8a7a4f1fbb707cb8"
format_stamp: "Formatted at 2026-09-26 11:40:23 on dist-test-slave-xxxx"
server_key: "52858f146bf3484caa95486448c81a88"
server_key_iv: "dc2882cddcc67152ecb670d1b2dc973a"
server_key_version: "encryptionkey@0"
I20260926 11:40:23.033209 7612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "900e2cd538cf416c8a7a4f1fbb707cb8"
format_stamp: "Formatted at 2026-09-26 11:40:23 on dist-test-slave-xxxx"
server_key: "52858f146bf3484caa95486448c81a88"
server_key_iv: "dc2882cddcc67152ecb670d1b2dc973a"
server_key_version: "encryptionkey@0"
I20260926 11:40:23.036844 7612 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 11:40:23.039269 7627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:23.040439 7612 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:40:23.040549 7612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "900e2cd538cf416c8a7a4f1fbb707cb8"
format_stamp: "Formatted at 2026-09-26 11:40:23 on dist-test-slave-xxxx"
server_key: "52858f146bf3484caa95486448c81a88"
server_key_iv: "dc2882cddcc67152ecb670d1b2dc973a"
server_key_version: "encryptionkey@0"
I20260926 11:40:23.040664 7612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:23.051654 7612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:23.054468 7612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:23.054642 7612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:23.061951 7612 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:36643
I20260926 11:40:23.061962 7689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:36643 every 8 connection(s)
I20260926 11:40:23.062882 7612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:40:23.063362 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 7612
I20260926 11:40:23.063470 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:40:23.063700 6528 external_mini_cluster.cc:1428] Setting key 78afa53e41d9626680bf624e62e230a2
I20260926 11:40:23.065840 7690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:23.071138 7690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8: Bootstrap starting.
Sep 26 11:40:23 dist-test-slave-xxxx krb5kdc[7599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:23.073339 7690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:23.074040 7690 log.cc:824] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:23.075955 7690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8: No bootstrap required, opened a new log
I20260926 11:40:23.077361 7693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.065000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57178 (local address 127.6.96.62:36643)
0926 11:40:23.065410 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:23.065425 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:23.065460 (+ 35us) server_negotiation.cc:407] Connection header received
0926 11:40:23.066123 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:23.066141 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:23.066471 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:23.066788 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:23.067553 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.068564 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.069410 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.069877 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.070020 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.072445 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:23.072450 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:23.072463 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:23.072490 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:23.074757 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:23.075201 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:23.075207 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:23.075213 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:23.075317 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:23.075554 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:23.075557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:23.075559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:23.075946 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:23.076130 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:23.076381 (+ 251us) server_negotiation.cc:299] Negotiation successful
0926 11:40:23.076652 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":118,"threads_started":1}
I20260926 11:40:23.079147 7690 raft_consensus.cc:348] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "900e2cd538cf416c8a7a4f1fbb707cb8" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36643 } }
I20260926 11:40:23.079417 7690 raft_consensus.cc:374] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:23.079491 7690 raft_consensus.cc:729] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 900e2cd538cf416c8a7a4f1fbb707cb8, State: Initialized, Role: FOLLOWER
I20260926 11:40:23.079906 7690 consensus_queue.cc:260] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "900e2cd538cf416c8a7a4f1fbb707cb8" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36643 } }
I20260926 11:40:23.080034 7690 raft_consensus.cc:388] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:40:23.080134 7690 raft_consensus.cc:482] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:40:23.080240 7690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:23.081071 7690 raft_consensus.cc:504] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "900e2cd538cf416c8a7a4f1fbb707cb8" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36643 } }
I20260926 11:40:23.081382 7690 leader_election.cc:302] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 900e2cd538cf416c8a7a4f1fbb707cb8; no voters:
I20260926 11:40:23.081645 7690 leader_election.cc:288] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:40:23.081750 7695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:23.081964 7695 raft_consensus.cc:686] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8 [term 1 LEADER]: Becoming Leader. State: Replica: 900e2cd538cf416c8a7a4f1fbb707cb8, State: Running, Role: LEADER
I20260926 11:40:23.082329 7695 consensus_queue.cc:237] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "900e2cd538cf416c8a7a4f1fbb707cb8" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36643 } }
I20260926 11:40:23.082901 7690 sys_catalog.cc:564] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:40:23.084115 7697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "900e2cd538cf416c8a7a4f1fbb707cb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "900e2cd538cf416c8a7a4f1fbb707cb8" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36643 } } }
I20260926 11:40:23.084147 7696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 900e2cd538cf416c8a7a4f1fbb707cb8. Latest consensus state: current_term: 1 leader_uuid: "900e2cd538cf416c8a7a4f1fbb707cb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "900e2cd538cf416c8a7a4f1fbb707cb8" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36643 } } }
I20260926 11:40:23.084231 7697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:23.084240 7696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8 [sys.catalog]: This master's current role is: LEADER
W20260926 11:40:23.087499 7710 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 11:40:23.087574 7710 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 11:40:23.087648 7705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:40:23.088348 7705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:40:23.093659 7705 catalog_manager.cc:1347] Generated new cluster ID: 61725aa2ef1040bfb85c5b6f197b4b79
I20260926 11:40:23.093751 7705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:40:23.105830 7705 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:40:23.107089 7705 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:40:23.114586 7705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8: Generated new TSK 0
I20260926 11:40:23.115475 7705 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.96.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
I20260926 11:40:23.143297 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:36643
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:46851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:23.255584 7716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:23.255849 7716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:23.255892 7716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:23.259887 7716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:23.259944 7716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:23.259984 7716 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 11:40:23.260021 7716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:40:23.265168 7716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:46851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.96.62:36643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:23.266157 7716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:23.267122 7716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:23.276495 7721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:23.276489 7724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:23.276549 7722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:23.276934 7716 server_base.cc:1034] running on GCE node
I20260926 11:40:23.277361 7716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:23.278021 7716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:23.279189 7716 hybrid_clock.cc:648] HybridClock initialized: now 1790422823279170 us; error 33 us; skew 500 ppm
Sep 26 11:40:23 dist-test-slave-xxxx krb5kdc[7599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422823, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:23.281996 7716 init.cc:377] Logged in from keytab as kudu/127.6.96.1@KRBTEST.COM (short username kudu)
I20260926 11:40:23.282928 7716 webserver.cc:460] Webserver started at http://127.6.96.1:43249/ using document root <none> and password file <none>
I20260926 11:40:23.283581 7716 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:23.283643 7716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:23.283968 7716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:23.286186 7716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "d316b542b21d424a84a494d4218f039b"
format_stamp: "Formatted at 2026-09-26 11:40:23 on dist-test-slave-xxxx"
server_key: "db031aed7c43cb2889d4987408c92ffc"
server_key_iv: "b5142906541777fa6105ab932536936f"
server_key_version: "encryptionkey@0"
I20260926 11:40:23.286710 7716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "d316b542b21d424a84a494d4218f039b"
format_stamp: "Formatted at 2026-09-26 11:40:23 on dist-test-slave-xxxx"
server_key: "db031aed7c43cb2889d4987408c92ffc"
server_key_iv: "b5142906541777fa6105ab932536936f"
server_key_version: "encryptionkey@0"
I20260926 11:40:23.290413 7716 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 11:40:23.292747 7731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:23.293987 7716 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:23.294133 7716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "d316b542b21d424a84a494d4218f039b"
format_stamp: "Formatted at 2026-09-26 11:40:23 on dist-test-slave-xxxx"
server_key: "db031aed7c43cb2889d4987408c92ffc"
server_key_iv: "b5142906541777fa6105ab932536936f"
server_key_version: "encryptionkey@0"
I20260926 11:40:23.294247 7716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:23.309949 7716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:23.312887 7716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:23.313059 7716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:23.314491 7716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:23.314538 7716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:23.314575 7716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:23.314601 7716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 11:40:23 dist-test-slave-xxxx krb5kdc[7599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422823, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:23.324777 7716 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:44457
I20260926 11:40:23.324837 7846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:44457 every 8 connection(s)
I20260926 11:40:23.325994 7716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:40:23.328742 7693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.315327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:57023 (local address 127.6.96.62:36643)
0926 11:40:23.315446 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:23.315451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:23.316557 (+ 1106us) server_negotiation.cc:407] Connection header received
0926 11:40:23.317569 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:23.317573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:23.317629 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:23.317716 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:23.319389 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.319985 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.321690 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.322041 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.322150 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.325104 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:23.325108 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:23.325111 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:23.325141 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:23.326680 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:23.327231 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:23.327234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:23.327236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:23.327321 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:23.327638 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:23.327641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:23.327643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:23.327841 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:23.327928 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:23.328445 (+ 517us) server_negotiation.cc:299] Negotiation successful
0926 11:40:23.328577 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 11:40:23.329303 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 7716
I20260926 11:40:23.329406 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:40:23.329579 7740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.315871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:36643 (local address 127.6.96.1:57023)
0926 11:40:23.316387 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:23.316446 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:23.317359 (+ 913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:23.317952 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:23.317963 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:23.318414 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:23.319192 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.319215 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.320093 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.320097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.321518 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.321529 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.322232 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.322236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.322434 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.323106 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:23.323128 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:23.324903 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:23.326822 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:23.326854 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:23.326874 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:23.327128 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:23.327402 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:23.327405 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:23.327407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:23.327540 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:23.327940 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:23.327947 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:23.328206 (+ 259us) client_negotiation.cc:769] Sending connection context
0926 11:40:23.328362 (+ 156us) client_negotiation.cc:240] Negotiation successful
0926 11:40:23.328697 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":129,"threads_started":1}
I20260926 11:40:23.329641 6528 external_mini_cluster.cc:1428] Setting key f12930c75669e102a3feb25e22e305d6
I20260926 11:40:23.330920 7847 heartbeater.cc:347] Connected to a master server at 127.6.96.62:36643
I20260926 11:40:23.331086 7847 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:23.331737 7847 heartbeater.cc:511] Master 127.6.96.62:36643 requested a full tablet report, sending...
I20260926 11:40:23.333513 7642 ts_manager.cc:194] Registered new tserver with Master: d316b542b21d424a84a494d4218f039b (127.6.96.1:44457)
I20260926 11:40:23.334993 7642 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.1@KRBTEST.COM'} at 127.6.96.1:57023
WARNING: no policy specified for kudu/127.6.96.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.2@KRBTEST.COM" created.
I20260926 11:40:23.342747 7854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.333172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:36643 (local address 127.6.96.1:58871)
0926 11:40:23.335067 (+ 1895us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:23.335081 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:23.335244 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:23.335564 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:23.335566 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:23.335815 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:23.335929 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.335941 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.336992 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.336996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.337904 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.337915 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.338366 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.338370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.338464 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.339030 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:23.339046 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:23.339439 (+ 393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:23.341048 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:23.341051 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:23.341052 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:23.341249 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:23.341617 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:23.341620 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:23.341621 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:23.341758 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:23.342189 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:23.342193 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:23.342307 (+ 114us) client_negotiation.cc:769] Sending connection context
0926 11:40:23.342430 (+ 123us) client_negotiation.cc:240] Negotiation successful
0926 11:40:23.342582 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1803,"thread_start_us":117,"threads_started":1}
I20260926 11:40:23.342861 7693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.333253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:58871 (local address 127.6.96.62:36643)
0926 11:40:23.333372 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:23.333377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:23.335164 (+ 1787us) server_negotiation.cc:407] Connection header received
0926 11:40:23.335361 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:23.335364 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:23.335411 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:23.335452 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:23.336075 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.336838 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.338024 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.338252 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.338334 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.339574 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:23.339578 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:23.339580 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:23.339608 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:23.340933 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:23.341403 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:23.341407 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:23.341409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:23.341528 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:23.341842 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:23.341844 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:23.341845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:23.342073 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:23.342157 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:23.342527 (+ 370us) server_negotiation.cc:299] Negotiation successful
0926 11:40:23.342666 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
I20260926 11:40:23.355858 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:36643
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:46851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:23.464088 7856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:23.464353 7856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:23.464397 7856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:23.468309 7856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:23.468365 7856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:23.468405 7856 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 11:40:23.468436 7856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:40:23.473556 7856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:46851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.96.62:36643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:23.474488 7856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:23.475492 7856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:23.484782 7864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:23.484802 7862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:23.484792 7861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:23.485173 7856 server_base.cc:1034] running on GCE node
I20260926 11:40:23.485589 7856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:23.486232 7856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:23.487391 7856 hybrid_clock.cc:648] HybridClock initialized: now 1790422823487371 us; error 31 us; skew 500 ppm
Sep 26 11:40:23 dist-test-slave-xxxx krb5kdc[7599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422823, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:23.490181 7856 init.cc:377] Logged in from keytab as kudu/127.6.96.2@KRBTEST.COM (short username kudu)
I20260926 11:40:23.491202 7856 webserver.cc:460] Webserver started at http://127.6.96.2:33757/ using document root <none> and password file <none>
I20260926 11:40:23.491914 7856 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:23.491998 7856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:23.492183 7856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:23.494323 7856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "0ef370ad8a444940909f329391e4a540"
format_stamp: "Formatted at 2026-09-26 11:40:23 on dist-test-slave-xxxx"
server_key: "ce6c061b2eda83099b34675562b959e5"
server_key_iv: "919043cc81225dd622ce165bc2bd6da8"
server_key_version: "encryptionkey@0"
I20260926 11:40:23.494892 7856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "0ef370ad8a444940909f329391e4a540"
format_stamp: "Formatted at 2026-09-26 11:40:23 on dist-test-slave-xxxx"
server_key: "ce6c061b2eda83099b34675562b959e5"
server_key_iv: "919043cc81225dd622ce165bc2bd6da8"
server_key_version: "encryptionkey@0"
I20260926 11:40:23.498766 7856 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260926 11:40:23.501223 7871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:23.502564 7856 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 11:40:23.502686 7856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "0ef370ad8a444940909f329391e4a540"
format_stamp: "Formatted at 2026-09-26 11:40:23 on dist-test-slave-xxxx"
server_key: "ce6c061b2eda83099b34675562b959e5"
server_key_iv: "919043cc81225dd622ce165bc2bd6da8"
server_key_version: "encryptionkey@0"
I20260926 11:40:23.502806 7856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:23.518414 7856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:23.521095 7856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:23.521236 7856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:23.522565 7856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:23.522619 7856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:23.522670 7856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:23.522696 7856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 11:40:23 dist-test-slave-xxxx krb5kdc[7599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422823, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:23.532612 7856 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:38389
I20260926 11:40:23.532714 7986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:38389 every 8 connection(s)
I20260926 11:40:23.533595 7856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:40:23.535983 7693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.523259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:56257 (local address 127.6.96.62:36643)
0926 11:40:23.523386 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:23.523389 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:23.524875 (+ 1486us) server_negotiation.cc:407] Connection header received
0926 11:40:23.526003 (+ 1128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:23.526006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:23.526061 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:23.526134 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:23.527616 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.528247 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.529437 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.529765 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.529855 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.532744 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:23.532747 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:23.532750 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:23.532778 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:23.534104 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:23.534650 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:23.534653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:23.534655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:23.534736 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:23.535022 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:23.535024 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:23.535025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:23.535204 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:23.535280 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:23.535758 (+ 478us) server_negotiation.cc:299] Negotiation successful
0926 11:40:23.535866 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 11:40:23.536691 7881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.524271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:36643 (local address 127.6.96.2:56257)
0926 11:40:23.524726 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:23.524767 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:23.525796 (+ 1029us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:23.526331 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:23.526341 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:23.526787 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:23.527461 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.527481 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.528362 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.528365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.529321 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.529330 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.529899 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.529903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.530071 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.530721 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:23.530742 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:23.532553 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:23.534235 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:23.534241 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:23.534255 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:23.534553 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:23.534803 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:23.534805 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:23.534807 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:23.534954 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:23.535282 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:23.535286 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:23.535501 (+ 215us) client_negotiation.cc:769] Sending connection context
0926 11:40:23.535642 (+ 141us) client_negotiation.cc:240] Negotiation successful
0926 11:40:23.535880 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":121,"threads_started":1}
I20260926 11:40:23.537932 7987 heartbeater.cc:347] Connected to a master server at 127.6.96.62:36643
I20260926 11:40:23.538090 7987 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:23.538704 7987 heartbeater.cc:511] Master 127.6.96.62:36643 requested a full tablet report, sending...
I20260926 11:40:23.539770 7643 ts_manager.cc:194] Registered new tserver with Master: 0ef370ad8a444940909f329391e4a540 (127.6.96.2:38389)
I20260926 11:40:23.540534 7643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.2@KRBTEST.COM'} at 127.6.96.2:56257
I20260926 11:40:23.541495 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 7856
I20260926 11:40:23.541608 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:40:23.541898 6528 external_mini_cluster.cc:1428] Setting key e4462c3104f0a923b11e4d7f489373cf
I20260926 11:40:23.547101 7993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.540123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:36643 (local address 127.6.96.2:53015)
0926 11:40:23.540469 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:23.540482 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:23.540571 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:23.540794 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:23.540796 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:23.540978 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:23.541075 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.541080 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.541803 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.541806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.542427 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.542439 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.542962 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.542965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.543046 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.543554 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:23.543565 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:23.543905 (+ 340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:23.545721 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:23.545724 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:23.545725 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:23.545926 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:23.546180 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:23.546183 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:23.546185 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:23.546255 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:23.546657 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:23.546658 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:23.546741 (+ 83us) client_negotiation.cc:769] Sending connection context
0926 11:40:23.546813 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 11:40:23.546961 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":104,"threads_started":1}
I20260926 11:40:23.547163 7693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.540194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:53015 (local address 127.6.96.62:36643)
0926 11:40:23.540326 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:23.540330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:23.540559 (+ 229us) server_negotiation.cc:407] Connection header received
0926 11:40:23.540661 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:23.540664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:23.540705 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:23.540769 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:23.541176 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.541684 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.542538 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.542865 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.542959 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.544044 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:23.544048 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:23.544051 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:23.544072 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:23.545603 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:23.546017 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:23.546021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:23.546023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:23.546103 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:23.546323 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:23.546326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:23.546328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:23.546582 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:23.546673 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:23.546901 (+ 228us) server_negotiation.cc:299] Negotiation successful
0926 11:40:23.547018 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
WARNING: no policy specified for kudu/127.6.96.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
I20260926 11:40:23.569749 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:36643
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:46851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:23.678889 7996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:23.679173 7996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:23.679220 7996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:23.683388 7996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:23.683459 7996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:23.683501 7996 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 11:40:23.683535 7996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:40:23.689397 7996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:46851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.96.62:36643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:23.690470 7996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:23.691519 7996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:23.700760 8002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:23.700762 8001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:23.700788 8004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:23.700878 7996 server_base.cc:1034] running on GCE node
I20260926 11:40:23.701465 7996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:23.702050 7996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:23.703253 7996 hybrid_clock.cc:648] HybridClock initialized: now 1790422823703232 us; error 38 us; skew 500 ppm
Sep 26 11:40:23 dist-test-slave-xxxx krb5kdc[7599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422823, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:23.705833 7996 init.cc:377] Logged in from keytab as kudu/127.6.96.3@KRBTEST.COM (short username kudu)
I20260926 11:40:23.706679 7996 webserver.cc:460] Webserver started at http://127.6.96.3:36625/ using document root <none> and password file <none>
I20260926 11:40:23.707226 7996 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:23.707285 7996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:23.707448 7996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:23.709400 7996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "f67d214e4b6f4a3a88ba0810ec332232"
format_stamp: "Formatted at 2026-09-26 11:40:23 on dist-test-slave-xxxx"
server_key: "e40fe4f9162539819a1760d995985d42"
server_key_iv: "c0b352bdb6b56c67dfa8d47f50b9bbf7"
server_key_version: "encryptionkey@0"
I20260926 11:40:23.709889 7996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "f67d214e4b6f4a3a88ba0810ec332232"
format_stamp: "Formatted at 2026-09-26 11:40:23 on dist-test-slave-xxxx"
server_key: "e40fe4f9162539819a1760d995985d42"
server_key_iv: "c0b352bdb6b56c67dfa8d47f50b9bbf7"
server_key_version: "encryptionkey@0"
I20260926 11:40:23.713578 7996 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 11:40:23.716238 8011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:23.717569 7996 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 11:40:23.717682 7996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "f67d214e4b6f4a3a88ba0810ec332232"
format_stamp: "Formatted at 2026-09-26 11:40:23 on dist-test-slave-xxxx"
server_key: "e40fe4f9162539819a1760d995985d42"
server_key_iv: "c0b352bdb6b56c67dfa8d47f50b9bbf7"
server_key_version: "encryptionkey@0"
I20260926 11:40:23.717799 7996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:23.731739 7996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:23.734666 7996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:23.734853 7996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:23.736202 7996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:23.736251 7996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:23.736301 7996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:23.736328 7996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 11:40:23 dist-test-slave-xxxx krb5kdc[7599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422823, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:23.746471 7996 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:36519
I20260926 11:40:23.746490 8126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:36519 every 8 connection(s)
I20260926 11:40:23.747414 7996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:40:23.748975 7693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.736985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:56087 (local address 127.6.96.62:36643)
0926 11:40:23.737130 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:23.737134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:23.738179 (+ 1045us) server_negotiation.cc:407] Connection header received
0926 11:40:23.739247 (+ 1068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:23.739251 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:23.739307 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:23.739380 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:23.740628 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.741150 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.742330 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.742555 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.742646 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.745627 (+ 2981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:23.745630 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:23.745632 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:23.745656 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:23.747097 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:23.747566 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:23.747570 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:23.747572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:23.747652 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:23.748010 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:23.748013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:23.748015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:23.748237 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:23.748315 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:23.748758 (+ 443us) server_negotiation.cc:299] Negotiation successful
0926 11:40:23.748858 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 11:40:23.749792 8020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.737563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:36643 (local address 127.6.96.3:56087)
0926 11:40:23.738024 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:23.738080 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:23.739037 (+ 957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:23.739660 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:23.739667 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:23.739973 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:23.740481 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.740497 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.741264 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.741267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.742208 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.742218 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.742704 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.742709 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.742950 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.743615 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:23.743638 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:23.745452 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:23.747199 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:23.747204 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:23.747219 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:23.747459 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:23.747735 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:23.747739 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:23.747741 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:23.747903 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:23.748332 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:23.748337 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:23.748549 (+ 212us) client_negotiation.cc:769] Sending connection context
0926 11:40:23.748706 (+ 157us) client_negotiation.cc:240] Negotiation successful
0926 11:40:23.748985 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":128,"threads_started":1}
I20260926 11:40:23.751042 8127 heartbeater.cc:347] Connected to a master server at 127.6.96.62:36643
I20260926 11:40:23.751173 8127 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:23.751667 8127 heartbeater.cc:511] Master 127.6.96.62:36643 requested a full tablet report, sending...
I20260926 11:40:23.752661 7642 ts_manager.cc:194] Registered new tserver with Master: f67d214e4b6f4a3a88ba0810ec332232 (127.6.96.3:36519)
I20260926 11:40:23.753345 7642 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.3@KRBTEST.COM'} at 127.6.96.3:56087
I20260926 11:40:23.755586 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 7996
I20260926 11:40:23.755685 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:40:23.755920 6528 external_mini_cluster.cc:1428] Setting key ce25ced33c0f13abb03d4af3bfb27768
I20260926 11:40:23.758040 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 11:40:23.760344 7693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.753109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:47165 (local address 127.6.96.62:36643)
0926 11:40:23.753220 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:23.753224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:23.753365 (+ 141us) server_negotiation.cc:407] Connection header received
0926 11:40:23.753512 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:23.753514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:23.753557 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:23.753631 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:23.754012 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.754507 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.755319 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.755529 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.755610 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.756843 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:23.756847 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:23.756849 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:23.756873 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:23.758775 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:23.759210 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:23.759214 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:23.759216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:23.759305 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:23.759550 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:23.759553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:23.759555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:23.759740 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:23.759828 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:23.760083 (+ 255us) server_negotiation.cc:299] Negotiation successful
0926 11:40:23.760182 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 11:40:23.760355 8133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.752998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:36643 (local address 127.6.96.3:47165)
0926 11:40:23.753291 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:23.753305 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:23.753396 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:23.753649 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:23.753652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:23.753820 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:23.753910 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.753916 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.754629 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.754632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.755219 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.755227 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.755634 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.755637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.755724 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.756282 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:23.756315 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:23.756706 (+ 391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:23.758924 (+ 2218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:23.758927 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:23.758929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:23.759122 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:23.759387 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:23.759390 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:23.759392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:23.759471 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:23.759828 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:23.759831 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:23.759933 (+ 102us) client_negotiation.cc:769] Sending connection context
0926 11:40:23.760018 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 11:40:23.760165 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":138,"threads_started":1}
Sep 26 11:40:23 dist-test-slave-xxxx krb5kdc[7599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422823, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 11:40:23 dist-test-slave-xxxx krb5kdc[7599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422823, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:23.782460 7693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.774882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57192 (local address 127.6.96.62:36643)
0926 11:40:23.775003 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:23.775006 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:23.775017 (+ 11us) server_negotiation.cc:407] Connection header received
0926 11:40:23.775089 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:23.775091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:23.775130 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:23.775199 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:23.775873 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.776392 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.777212 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.777428 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.777523 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.779863 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:23.779867 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:23.779869 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:23.779895 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:23.781043 (+ 1148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:23.781458 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:23.781461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:23.781462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:23.781538 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:23.781751 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:23.781753 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:23.781754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:23.781920 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:23.782015 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:23.782219 (+ 204us) server_negotiation.cc:299] Negotiation successful
0926 11:40:23.782333 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 11:40:23.785336 7642 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57192:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 11:40:23.788472 7642 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:40:23.802162 8148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.797314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59206 (local address 127.6.96.1:44457)
0926 11:40:23.797725 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:23.797739 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:23.797763 (+ 24us) server_negotiation.cc:407] Connection header received
0926 11:40:23.797850 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:23.797854 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:23.798037 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:23.798458 (+ 421us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:23.798567 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.799497 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.800853 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.801521 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.801616 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.801780 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:23.801926 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 11:40:23.802009 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":86,"threads_started":1}
I20260926 11:40:23.802153 8147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.796636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:44457 (local address 127.0.0.1:59206)
0926 11:40:23.797391 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:23.797404 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:23.797482 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:23.798185 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:23.798188 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:23.798206 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:23.798293 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.798298 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.799609 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.799613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.800732 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.800741 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.801635 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.801639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.801740 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.801778 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 11:40:23.801886 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 11:40:23.802006 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"mutex_wait_us":77,"thread_start_us":91,"threads_started":1}
I20260926 11:40:23.802424 8146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.797175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:36519 (local address 127.0.0.1:36292)
0926 11:40:23.797663 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:23.797675 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:23.797743 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:23.798624 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:23.798627 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:23.798644 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:23.798726 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.798732 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.799934 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.799938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.801037 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.801045 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.802040 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.802043 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.802141 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.802159 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 11:40:23.802199 (+ 40us) client_negotiation.cc:240] Negotiation successful
0926 11:40:23.802263 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":5,"thread_start_us":87,"threads_started":1}
I20260926 11:40:23.802424 8144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.796449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:38389 (local address 127.0.0.1:56866)
0926 11:40:23.796796 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:23.796847 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:23.796976 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:23.797653 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:23.797658 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:23.797681 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:23.797822 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.797829 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.799362 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.799365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.800728 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.800738 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.801993 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.801997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.802082 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.802141 (+ 59us) client_negotiation.cc:769] Sending connection context
0926 11:40:23.802210 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 11:40:23.802263 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":34,"thread_start_us":76,"threads_started":1}
I20260926 11:40:23.802773 8145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.796600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56866 (local address 127.6.96.2:38389)
0926 11:40:23.797083 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:23.797095 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:23.797118 (+ 23us) server_negotiation.cc:407] Connection header received
0926 11:40:23.797190 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:23.797196 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:23.797407 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:23.797559 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:23.797998 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.799242 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.800863 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.801676 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.801775 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.801931 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:23.802538 (+ 607us) server_negotiation.cc:299] Negotiation successful
0926 11:40:23.802638 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":101,"threads_started":1}
I20260926 11:40:23.802907 8149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.797904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36292 (local address 127.6.96.3:36519)
0926 11:40:23.798216 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:23.798226 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:23.798247 (+ 21us) server_negotiation.cc:407] Connection header received
0926 11:40:23.798311 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:23.798316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:23.798474 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:23.798708 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:23.798851 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.799822 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.801161 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.801929 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.802478 (+ 549us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.802632 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:23.802686 (+ 54us) server_negotiation.cc:299] Negotiation successful
0926 11:40:23.802762 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":105,"threads_started":1}
I20260926 11:40:23.805415 7781 tablet_service.cc:1467] Processing CreateTablet for tablet 3e4f8041ebd4419fb70875831c574090 (DEFAULT_TABLE table=test-table [id=da471101a2a94b6591257a2cb448a1f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:23.805476 7921 tablet_service.cc:1467] Processing CreateTablet for tablet 3e4f8041ebd4419fb70875831c574090 (DEFAULT_TABLE table=test-table [id=da471101a2a94b6591257a2cb448a1f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:23.805476 8061 tablet_service.cc:1467] Processing CreateTablet for tablet 3e4f8041ebd4419fb70875831c574090 (DEFAULT_TABLE table=test-table [id=da471101a2a94b6591257a2cb448a1f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:23.806427 7781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e4f8041ebd4419fb70875831c574090. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:23.806463 7921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e4f8041ebd4419fb70875831c574090. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:23.806468 8061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e4f8041ebd4419fb70875831c574090. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:23.813231 8150 tablet_bootstrap.cc:492] T 3e4f8041ebd4419fb70875831c574090 P d316b542b21d424a84a494d4218f039b: Bootstrap starting.
I20260926 11:40:23.814939 8152 tablet_bootstrap.cc:492] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232: Bootstrap starting.
I20260926 11:40:23.815152 8151 tablet_bootstrap.cc:492] T 3e4f8041ebd4419fb70875831c574090 P 0ef370ad8a444940909f329391e4a540: Bootstrap starting.
I20260926 11:40:23.815773 8150 tablet_bootstrap.cc:654] T 3e4f8041ebd4419fb70875831c574090 P d316b542b21d424a84a494d4218f039b: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:23.816483 8150 log.cc:824] T 3e4f8041ebd4419fb70875831c574090 P d316b542b21d424a84a494d4218f039b: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:23.817088 8151 tablet_bootstrap.cc:654] T 3e4f8041ebd4419fb70875831c574090 P 0ef370ad8a444940909f329391e4a540: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:23.817088 8152 tablet_bootstrap.cc:654] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:23.817911 8151 log.cc:824] T 3e4f8041ebd4419fb70875831c574090 P 0ef370ad8a444940909f329391e4a540: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:23.817911 8152 log.cc:824] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:23.818688 8150 tablet_bootstrap.cc:492] T 3e4f8041ebd4419fb70875831c574090 P d316b542b21d424a84a494d4218f039b: No bootstrap required, opened a new log
I20260926 11:40:23.818905 8150 ts_tablet_manager.cc:1397] T 3e4f8041ebd4419fb70875831c574090 P d316b542b21d424a84a494d4218f039b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 11:40:23.819983 8152 tablet_bootstrap.cc:492] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232: No bootstrap required, opened a new log
I20260926 11:40:23.820029 8151 tablet_bootstrap.cc:492] T 3e4f8041ebd4419fb70875831c574090 P 0ef370ad8a444940909f329391e4a540: No bootstrap required, opened a new log
I20260926 11:40:23.820168 8151 ts_tablet_manager.cc:1397] T 3e4f8041ebd4419fb70875831c574090 P 0ef370ad8a444940909f329391e4a540: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 11:40:23.820168 8152 ts_tablet_manager.cc:1397] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260926 11:40:23.822090 8150 raft_consensus.cc:348] T 3e4f8041ebd4419fb70875831c574090 P d316b542b21d424a84a494d4218f039b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:23.822355 8150 raft_consensus.cc:374] T 3e4f8041ebd4419fb70875831c574090 P d316b542b21d424a84a494d4218f039b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:23.822408 8150 raft_consensus.cc:729] T 3e4f8041ebd4419fb70875831c574090 P d316b542b21d424a84a494d4218f039b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d316b542b21d424a84a494d4218f039b, State: Initialized, Role: FOLLOWER
I20260926 11:40:23.822840 8150 consensus_queue.cc:260] T 3e4f8041ebd4419fb70875831c574090 P d316b542b21d424a84a494d4218f039b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:23.823709 8150 ts_tablet_manager.cc:1428] T 3e4f8041ebd4419fb70875831c574090 P d316b542b21d424a84a494d4218f039b: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260926 11:40:23.824049 7847 heartbeater.cc:503] Master 127.6.96.62:36643 was elected leader, sending a full tablet report...
I20260926 11:40:23.824015 8152 raft_consensus.cc:348] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:23.824015 8151 raft_consensus.cc:348] T 3e4f8041ebd4419fb70875831c574090 P 0ef370ad8a444940909f329391e4a540 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:23.824249 8152 raft_consensus.cc:374] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:23.824259 8151 raft_consensus.cc:374] T 3e4f8041ebd4419fb70875831c574090 P 0ef370ad8a444940909f329391e4a540 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:23.824298 8151 raft_consensus.cc:729] T 3e4f8041ebd4419fb70875831c574090 P 0ef370ad8a444940909f329391e4a540 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ef370ad8a444940909f329391e4a540, State: Initialized, Role: FOLLOWER
I20260926 11:40:23.824304 8152 raft_consensus.cc:729] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f67d214e4b6f4a3a88ba0810ec332232, State: Initialized, Role: FOLLOWER
I20260926 11:40:23.824721 8151 consensus_queue.cc:260] T 3e4f8041ebd4419fb70875831c574090 P 0ef370ad8a444940909f329391e4a540 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:23.824721 8152 consensus_queue.cc:260] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:23.825675 8151 ts_tablet_manager.cc:1428] T 3e4f8041ebd4419fb70875831c574090 P 0ef370ad8a444940909f329391e4a540: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 11:40:23.825675 8152 ts_tablet_manager.cc:1428] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 11:40:23.826011 7987 heartbeater.cc:503] Master 127.6.96.62:36643 was elected leader, sending a full tablet report...
I20260926 11:40:23.826547 8127 heartbeater.cc:503] Master 127.6.96.62:36643 was elected leader, sending a full tablet report...
W20260926 11:40:23.827167 7848 tablet.cc:2369] T 3e4f8041ebd4419fb70875831c574090 P d316b542b21d424a84a494d4218f039b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:40:23.833971 8158 raft_consensus.cc:482] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:40:23.834158 8158 raft_consensus.cc:504] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:23.835335 8158 leader_election.cc:288] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ef370ad8a444940909f329391e4a540 (127.6.96.2:38389), d316b542b21d424a84a494d4218f039b (127.6.96.1:44457)
I20260926 11:40:23.838709 8020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.835536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:44457 (local address 127.6.96.3:32965)
0926 11:40:23.835680 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:23.835695 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:23.835806 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:23.836029 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:23.836032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:23.836047 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:23.836151 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.836156 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.836834 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.836838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.837625 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.837636 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.838292 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.838295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.838414 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.838431 (+ 17us) client_negotiation.cc:769] Sending connection context
0926 11:40:23.838493 (+ 62us) client_negotiation.cc:240] Negotiation successful
0926 11:40:23.838565 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":1}
I20260926 11:40:23.838711 8148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.835621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:32965 (local address 127.6.96.1:44457)
0926 11:40:23.835724 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:23.835729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:23.835742 (+ 13us) server_negotiation.cc:407] Connection header received
0926 11:40:23.835906 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:23.835908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:23.835950 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:23.836010 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:23.836243 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.836736 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.837755 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.838174 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.838278 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.838341 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:23.838516 (+ 175us) server_negotiation.cc:299] Negotiation successful
0926 11:40:23.838586 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 11:40:23.839422 7801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e4f8041ebd4419fb70875831c574090" candidate_uuid: "f67d214e4b6f4a3a88ba0810ec332232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d316b542b21d424a84a494d4218f039b" is_pre_election: true
I20260926 11:40:23.839592 8145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.835663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:56343 (local address 127.6.96.2:38389)
0926 11:40:23.835768 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:23.835772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:23.836114 (+ 342us) server_negotiation.cc:407] Connection header received
0926 11:40:23.836208 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:23.836211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:23.836255 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:23.836314 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:23.836548 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.837035 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.838490 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.839112 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.839202 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.839257 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:23.839431 (+ 174us) server_negotiation.cc:299] Negotiation successful
0926 11:40:23.839495 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 11:40:23.839615 8159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.835536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:38389 (local address 127.6.96.3:56343)
0926 11:40:23.836028 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:23.836038 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:23.836130 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:23.836327 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:23.836330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:23.836344 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:23.836458 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.836462 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.837130 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.837133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.838386 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.838395 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.839229 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.839232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.839323 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.839345 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 11:40:23.839406 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 11:40:23.839471 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":191,"thread_start_us":123,"threads_started":1}
I20260926 11:40:23.839740 7801 raft_consensus.cc:2454] T 3e4f8041ebd4419fb70875831c574090 P d316b542b21d424a84a494d4218f039b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f67d214e4b6f4a3a88ba0810ec332232 in term 0.
I20260926 11:40:23.840118 7941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e4f8041ebd4419fb70875831c574090" candidate_uuid: "f67d214e4b6f4a3a88ba0810ec332232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef370ad8a444940909f329391e4a540" is_pre_election: true
I20260926 11:40:23.840246 8013 leader_election.cc:302] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d316b542b21d424a84a494d4218f039b, f67d214e4b6f4a3a88ba0810ec332232; no voters:
I20260926 11:40:23.840397 7941 raft_consensus.cc:2454] T 3e4f8041ebd4419fb70875831c574090 P 0ef370ad8a444940909f329391e4a540 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f67d214e4b6f4a3a88ba0810ec332232 in term 0.
I20260926 11:40:23.840509 8158 raft_consensus.cc:2781] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:40:23.840602 8158 raft_consensus.cc:482] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:40:23.840654 8158 raft_consensus.cc:3037] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:23.841637 8158 raft_consensus.cc:504] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:23.841970 8158 leader_election.cc:288] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232 [CANDIDATE]: Term 1 election: Requested vote from peers 0ef370ad8a444940909f329391e4a540 (127.6.96.2:38389), d316b542b21d424a84a494d4218f039b (127.6.96.1:44457)
I20260926 11:40:23.842284 7941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e4f8041ebd4419fb70875831c574090" candidate_uuid: "f67d214e4b6f4a3a88ba0810ec332232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef370ad8a444940909f329391e4a540"
I20260926 11:40:23.842399 7941 raft_consensus.cc:3037] T 3e4f8041ebd4419fb70875831c574090 P 0ef370ad8a444940909f329391e4a540 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:23.842418 7801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e4f8041ebd4419fb70875831c574090" candidate_uuid: "f67d214e4b6f4a3a88ba0810ec332232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d316b542b21d424a84a494d4218f039b"
I20260926 11:40:23.842531 7801 raft_consensus.cc:3037] T 3e4f8041ebd4419fb70875831c574090 P d316b542b21d424a84a494d4218f039b [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:23.843694 7801 raft_consensus.cc:2454] T 3e4f8041ebd4419fb70875831c574090 P d316b542b21d424a84a494d4218f039b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f67d214e4b6f4a3a88ba0810ec332232 in term 1.
I20260926 11:40:23.843885 7941 raft_consensus.cc:2454] T 3e4f8041ebd4419fb70875831c574090 P 0ef370ad8a444940909f329391e4a540 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f67d214e4b6f4a3a88ba0810ec332232 in term 1.
I20260926 11:40:23.843979 8013 leader_election.cc:302] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d316b542b21d424a84a494d4218f039b, f67d214e4b6f4a3a88ba0810ec332232; no voters:
I20260926 11:40:23.844149 8158 raft_consensus.cc:2781] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:23.844347 8158 raft_consensus.cc:686] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232 [term 1 LEADER]: Becoming Leader. State: Replica: f67d214e4b6f4a3a88ba0810ec332232, State: Running, Role: LEADER
I20260926 11:40:23.844682 8158 consensus_queue.cc:237] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:23.847860 7643 catalog_manager.cc:5495] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232 reported cstate change: term changed from 0 to 1, leader changed from <none> to f67d214e4b6f4a3a88ba0810ec332232 (127.6.96.3). New cstate: current_term: 1 leader_uuid: "f67d214e4b6f4a3a88ba0810ec332232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } health_report { overall_health: HEALTHY } } }
I20260926 11:40:23.870675 8149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.866887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36302 (local address 127.6.96.3:36519)
0926 11:40:23.867012 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:23.867017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:23.867049 (+ 32us) server_negotiation.cc:407] Connection header received
0926 11:40:23.867279 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:23.867282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:23.867348 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:23.867435 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:40:23.867704 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.868233 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.869153 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.869383 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.869472 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.869841 (+ 369us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:40:23.870272 (+ 431us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:40:23.870376 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:23.870469 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 11:40:23.870530 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 11:40:23.876847 7801 raft_consensus.cc:1261] T 3e4f8041ebd4419fb70875831c574090 P d316b542b21d424a84a494d4218f039b [term 1 FOLLOWER]: Refusing update from remote peer f67d214e4b6f4a3a88ba0810ec332232: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:23.876847 7941 raft_consensus.cc:1261] T 3e4f8041ebd4419fb70875831c574090 P 0ef370ad8a444940909f329391e4a540 [term 1 FOLLOWER]: Refusing update from remote peer f67d214e4b6f4a3a88ba0810ec332232: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:23.877486 8160 consensus_queue.cc:1035] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:23.877707 8158 consensus_queue.cc:1035] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:23.897403 7643 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:23.897573 7643 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:23.900054 7643 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8: Sending DeleteTablet for 3 replicas of tablet 3e4f8041ebd4419fb70875831c574090
I20260926 11:40:23.900871 8061 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e4f8041ebd4419fb70875831c574090 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:23 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:36292
I20260926 11:40:23.900856 7921 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e4f8041ebd4419fb70875831c574090 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:23 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:56866
I20260926 11:40:23.901504 8174 tablet_replica.cc:330] T 3e4f8041ebd4419fb70875831c574090 P 0ef370ad8a444940909f329391e4a540: stopping tablet replica
I20260926 11:40:23.901499 7781 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e4f8041ebd4419fb70875831c574090 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:23 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:59206
I20260926 11:40:23.901551 7643 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57192:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 11:40:23.901945 8173 tablet_replica.cc:330] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232: stopping tablet replica
I20260926 11:40:23.901940 8175 tablet_replica.cc:330] T 3e4f8041ebd4419fb70875831c574090 P d316b542b21d424a84a494d4218f039b: stopping tablet replica
W20260926 11:40:23.901975 7643 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:40:23.902248 8173 raft_consensus.cc:2229] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232 [term 1 LEADER]: Raft consensus shutting down.
I20260926 11:40:23.902248 8175 raft_consensus.cc:2229] T 3e4f8041ebd4419fb70875831c574090 P d316b542b21d424a84a494d4218f039b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:23.902575 8175 raft_consensus.cc:2258] T 3e4f8041ebd4419fb70875831c574090 P d316b542b21d424a84a494d4218f039b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:23.903036 8173 raft_consensus.cc:2258] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:23.902593 8174 raft_consensus.cc:2229] T 3e4f8041ebd4419fb70875831c574090 P 0ef370ad8a444940909f329391e4a540 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:23.903479 8174 raft_consensus.cc:2258] T 3e4f8041ebd4419fb70875831c574090 P 0ef370ad8a444940909f329391e4a540 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:23.904615 8173 ts_tablet_manager.cc:1905] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:23.906776 7921 tablet_service.cc:1467] Processing CreateTablet for tablet 3df1522c79b649a5820960dc4bfea031 (DEFAULT_TABLE table=test-table [id=a9e196508749469a873319e76c215f38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:23.907069 7921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3df1522c79b649a5820960dc4bfea031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:23.907023 7781 tablet_service.cc:1467] Processing CreateTablet for tablet 3df1522c79b649a5820960dc4bfea031 (DEFAULT_TABLE table=test-table [id=a9e196508749469a873319e76c215f38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:23.907246 7781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3df1522c79b649a5820960dc4bfea031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:23.907426 8175 ts_tablet_manager.cc:1905] T 3e4f8041ebd4419fb70875831c574090 P d316b542b21d424a84a494d4218f039b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:23.907675 8174 ts_tablet_manager.cc:1905] T 3e4f8041ebd4419fb70875831c574090 P 0ef370ad8a444940909f329391e4a540: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:23.909242 8151 tablet_bootstrap.cc:492] T 3df1522c79b649a5820960dc4bfea031 P 0ef370ad8a444940909f329391e4a540: Bootstrap starting.
I20260926 11:40:23.909528 8175 ts_tablet_manager.cc:1918] T 3e4f8041ebd4419fb70875831c574090 P d316b542b21d424a84a494d4218f039b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:40:23.909612 8175 log.cc:1196] T 3e4f8041ebd4419fb70875831c574090 P d316b542b21d424a84a494d4218f039b: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-0/wal/wals/3e4f8041ebd4419fb70875831c574090
I20260926 11:40:23.909590 8061 tablet_service.cc:1467] Processing CreateTablet for tablet 3df1522c79b649a5820960dc4bfea031 (DEFAULT_TABLE table=test-table [id=a9e196508749469a873319e76c215f38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:23.909806 8061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3df1522c79b649a5820960dc4bfea031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:23.909979 8175 ts_tablet_manager.cc:1939] T 3e4f8041ebd4419fb70875831c574090 P d316b542b21d424a84a494d4218f039b: Deleting consensus metadata
I20260926 11:40:23.910133 8151 tablet_bootstrap.cc:654] T 3df1522c79b649a5820960dc4bfea031 P 0ef370ad8a444940909f329391e4a540: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:23.911053 8173 ts_tablet_manager.cc:1918] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:40:23.911165 8173 log.cc:1196] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/3e4f8041ebd4419fb70875831c574090
I20260926 11:40:23.911253 8151 tablet_bootstrap.cc:492] T 3df1522c79b649a5820960dc4bfea031 P 0ef370ad8a444940909f329391e4a540: No bootstrap required, opened a new log
I20260926 11:40:23.911331 8151 ts_tablet_manager.cc:1397] T 3df1522c79b649a5820960dc4bfea031 P 0ef370ad8a444940909f329391e4a540: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:23.911355 8174 ts_tablet_manager.cc:1918] T 3e4f8041ebd4419fb70875831c574090 P 0ef370ad8a444940909f329391e4a540: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:40:23.911419 8174 log.cc:1196] T 3e4f8041ebd4419fb70875831c574090 P 0ef370ad8a444940909f329391e4a540: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-1/wal/wals/3e4f8041ebd4419fb70875831c574090
I20260926 11:40:23.911461 8173 ts_tablet_manager.cc:1939] T 3e4f8041ebd4419fb70875831c574090 P f67d214e4b6f4a3a88ba0810ec332232: Deleting consensus metadata
I20260926 11:40:23.911690 8174 ts_tablet_manager.cc:1939] T 3e4f8041ebd4419fb70875831c574090 P 0ef370ad8a444940909f329391e4a540: Deleting consensus metadata
I20260926 11:40:23.911779 8151 raft_consensus.cc:348] T 3df1522c79b649a5820960dc4bfea031 P 0ef370ad8a444940909f329391e4a540 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:23.911875 8151 raft_consensus.cc:374] T 3df1522c79b649a5820960dc4bfea031 P 0ef370ad8a444940909f329391e4a540 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:23.911903 8151 raft_consensus.cc:729] T 3df1522c79b649a5820960dc4bfea031 P 0ef370ad8a444940909f329391e4a540 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ef370ad8a444940909f329391e4a540, State: Initialized, Role: FOLLOWER
I20260926 11:40:23.912050 8150 tablet_bootstrap.cc:492] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b: Bootstrap starting.
I20260926 11:40:23.912019 8151 consensus_queue.cc:260] T 3df1522c79b649a5820960dc4bfea031 P 0ef370ad8a444940909f329391e4a540 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:23.912309 8151 ts_tablet_manager.cc:1428] T 3df1522c79b649a5820960dc4bfea031 P 0ef370ad8a444940909f329391e4a540: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:23.912894 8150 tablet_bootstrap.cc:654] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:23.913615 8152 tablet_bootstrap.cc:492] T 3df1522c79b649a5820960dc4bfea031 P f67d214e4b6f4a3a88ba0810ec332232: Bootstrap starting.
I20260926 11:40:23.914417 8150 tablet_bootstrap.cc:492] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b: No bootstrap required, opened a new log
I20260926 11:40:23.914465 8152 tablet_bootstrap.cc:654] T 3df1522c79b649a5820960dc4bfea031 P f67d214e4b6f4a3a88ba0810ec332232: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:23.914495 8150 ts_tablet_manager.cc:1397] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:23.914913 8150 raft_consensus.cc:348] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:23.915025 8150 raft_consensus.cc:374] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:23.915059 8150 raft_consensus.cc:729] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d316b542b21d424a84a494d4218f039b, State: Initialized, Role: FOLLOWER
I20260926 11:40:23.915163 8150 consensus_queue.cc:260] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:23.915433 7629 catalog_manager.cc:4841] TS d316b542b21d424a84a494d4218f039b (127.6.96.1:44457): tablet 3e4f8041ebd4419fb70875831c574090 (table test-table [id=da471101a2a94b6591257a2cb448a1f7]) successfully deleted
I20260926 11:40:23.915439 7630 catalog_manager.cc:4841] TS 0ef370ad8a444940909f329391e4a540 (127.6.96.2:38389): tablet 3e4f8041ebd4419fb70875831c574090 (table test-table [id=da471101a2a94b6591257a2cb448a1f7]) successfully deleted
I20260926 11:40:23.915675 7629 catalog_manager.cc:4841] TS f67d214e4b6f4a3a88ba0810ec332232 (127.6.96.3:36519): tablet 3e4f8041ebd4419fb70875831c574090 (table test-table [id=da471101a2a94b6591257a2cb448a1f7]) successfully deleted
I20260926 11:40:23.915761 8152 tablet_bootstrap.cc:492] T 3df1522c79b649a5820960dc4bfea031 P f67d214e4b6f4a3a88ba0810ec332232: No bootstrap required, opened a new log
I20260926 11:40:23.915829 8152 ts_tablet_manager.cc:1397] T 3df1522c79b649a5820960dc4bfea031 P f67d214e4b6f4a3a88ba0810ec332232: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:23.916222 8152 raft_consensus.cc:348] T 3df1522c79b649a5820960dc4bfea031 P f67d214e4b6f4a3a88ba0810ec332232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:23.916316 8152 raft_consensus.cc:374] T 3df1522c79b649a5820960dc4bfea031 P f67d214e4b6f4a3a88ba0810ec332232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:23.916343 8152 raft_consensus.cc:729] T 3df1522c79b649a5820960dc4bfea031 P f67d214e4b6f4a3a88ba0810ec332232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f67d214e4b6f4a3a88ba0810ec332232, State: Initialized, Role: FOLLOWER
I20260926 11:40:23.915899 8150 ts_tablet_manager.cc:1428] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:23.916447 8152 consensus_queue.cc:260] T 3df1522c79b649a5820960dc4bfea031 P f67d214e4b6f4a3a88ba0810ec332232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:23.916688 8152 ts_tablet_manager.cc:1428] T 3df1522c79b649a5820960dc4bfea031 P f67d214e4b6f4a3a88ba0810ec332232: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:23.980737 8156 raft_consensus.cc:482] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:40:23.980896 8156 raft_consensus.cc:504] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:23.982205 8156 leader_election.cc:288] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ef370ad8a444940909f329391e4a540 (127.6.96.2:38389), f67d214e4b6f4a3a88ba0810ec332232 (127.6.96.3:36519)
I20260926 11:40:23.986778 8149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.982579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:36403 (local address 127.6.96.3:36519)
0926 11:40:23.982732 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:23.982737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:23.983118 (+ 381us) server_negotiation.cc:407] Connection header received
0926 11:40:23.983310 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:23.983313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:23.983376 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:23.983421 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:23.983911 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.984526 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.985627 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.986166 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.986271 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.986337 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:23.986555 (+ 218us) server_negotiation.cc:299] Negotiation successful
0926 11:40:23.986645 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 11:40:23.986881 8145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.982612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:34837 (local address 127.6.96.2:38389)
0926 11:40:23.982735 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:23.982740 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:23.983009 (+ 269us) server_negotiation.cc:407] Connection header received
0926 11:40:23.983136 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:23.983139 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:23.983181 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:23.983241 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:23.983513 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.984188 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.985797 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.986360 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.986452 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.986507 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:23.986678 (+ 171us) server_negotiation.cc:299] Negotiation successful
0926 11:40:23.986745 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 11:40:23.986896 8179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.982429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:38389 (local address 127.6.96.1:34837)
0926 11:40:23.982904 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:23.982918 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:23.983036 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:23.983289 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:23.983291 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:23.983307 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:23.983413 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.983418 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.984316 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.984319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.985672 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.985683 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.986469 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.986472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.986572 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.986594 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 11:40:23.986655 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 11:40:23.986731 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":82,"threads_started":1}
I20260926 11:40:23.987280 7941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3df1522c79b649a5820960dc4bfea031" candidate_uuid: "d316b542b21d424a84a494d4218f039b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef370ad8a444940909f329391e4a540" is_pre_election: true
I20260926 11:40:23.987421 7941 raft_consensus.cc:2454] T 3df1522c79b649a5820960dc4bfea031 P 0ef370ad8a444940909f329391e4a540 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d316b542b21d424a84a494d4218f039b in term 0.
I20260926 11:40:23.987612 8180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:23.982429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:36519 (local address 127.6.96.1:36403)
0926 11:40:23.983018 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:23.983032 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:23.983198 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:23.983560 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:23.983563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:23.983579 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:23.983756 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.983763 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.984640 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.984643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.985505 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:23.985515 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:23.986304 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:23.986308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:23.986407 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:23.986433 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 11:40:23.987383 (+ 950us) client_negotiation.cc:240] Negotiation successful
0926 11:40:23.987455 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"mutex_wait_us":250,"thread_start_us":142,"threads_started":1}
I20260926 11:40:23.987773 7734 leader_election.cc:302] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ef370ad8a444940909f329391e4a540, d316b542b21d424a84a494d4218f039b; no voters:
I20260926 11:40:23.988045 8156 raft_consensus.cc:2781] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:40:23.988097 8156 raft_consensus.cc:482] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:40:23.988131 8156 raft_consensus.cc:3037] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:23.988133 8081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3df1522c79b649a5820960dc4bfea031" candidate_uuid: "d316b542b21d424a84a494d4218f039b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f67d214e4b6f4a3a88ba0810ec332232" is_pre_election: true
I20260926 11:40:23.988315 8081 raft_consensus.cc:2454] T 3df1522c79b649a5820960dc4bfea031 P f67d214e4b6f4a3a88ba0810ec332232 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d316b542b21d424a84a494d4218f039b in term 0.
I20260926 11:40:23.989326 8156 raft_consensus.cc:504] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:23.989713 8156 leader_election.cc:288] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b [CANDIDATE]: Term 1 election: Requested vote from peers 0ef370ad8a444940909f329391e4a540 (127.6.96.2:38389), f67d214e4b6f4a3a88ba0810ec332232 (127.6.96.3:36519)
I20260926 11:40:23.990093 8081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3df1522c79b649a5820960dc4bfea031" candidate_uuid: "d316b542b21d424a84a494d4218f039b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f67d214e4b6f4a3a88ba0810ec332232"
I20260926 11:40:23.990226 8081 raft_consensus.cc:3037] T 3df1522c79b649a5820960dc4bfea031 P f67d214e4b6f4a3a88ba0810ec332232 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:23.990211 7941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3df1522c79b649a5820960dc4bfea031" candidate_uuid: "d316b542b21d424a84a494d4218f039b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef370ad8a444940909f329391e4a540"
I20260926 11:40:23.990335 7941 raft_consensus.cc:3037] T 3df1522c79b649a5820960dc4bfea031 P 0ef370ad8a444940909f329391e4a540 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:23.991379 8081 raft_consensus.cc:2454] T 3df1522c79b649a5820960dc4bfea031 P f67d214e4b6f4a3a88ba0810ec332232 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d316b542b21d424a84a494d4218f039b in term 1.
I20260926 11:40:23.991436 7941 raft_consensus.cc:2454] T 3df1522c79b649a5820960dc4bfea031 P 0ef370ad8a444940909f329391e4a540 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d316b542b21d424a84a494d4218f039b in term 1.
I20260926 11:40:23.991673 7733 leader_election.cc:302] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d316b542b21d424a84a494d4218f039b, f67d214e4b6f4a3a88ba0810ec332232; no voters:
I20260926 11:40:23.991886 8156 raft_consensus.cc:2781] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:23.992086 8156 raft_consensus.cc:686] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b [term 1 LEADER]: Becoming Leader. State: Replica: d316b542b21d424a84a494d4218f039b, State: Running, Role: LEADER
I20260926 11:40:23.992327 8156 consensus_queue.cc:237] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:23.994534 7642 catalog_manager.cc:5495] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b reported cstate change: term changed from 0 to 1, leader changed from <none> to d316b542b21d424a84a494d4218f039b (127.6.96.1). New cstate: current_term: 1 leader_uuid: "d316b542b21d424a84a494d4218f039b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } health_report { overall_health: UNKNOWN } } }
W20260926 11:40:23.998332 8128 tablet.cc:2369] T 3df1522c79b649a5820960dc4bfea031 P f67d214e4b6f4a3a88ba0810ec332232: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 11:40:24.034690 7988 tablet.cc:2369] T 3df1522c79b649a5820960dc4bfea031 P 0ef370ad8a444940909f329391e4a540: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 11:40:24 dist-test-slave-xxxx krb5kdc[7599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422823, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:24.041719 7693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:24.032922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57204 (local address 127.6.96.62:36643)
0926 11:40:24.033050 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:24.033055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:24.033269 (+ 214us) server_negotiation.cc:407] Connection header received
0926 11:40:24.033370 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:24.033373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:24.033420 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:24.033513 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:24.034114 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:24.034996 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:24.035826 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:24.036155 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:24.036249 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:24.038632 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:24.038635 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:24.038637 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:24.038659 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:24.039963 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:24.040471 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:24.040475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:24.040476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:24.040553 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:24.040835 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:24.040837 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:24.040839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:24.041015 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:24.041133 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:24.041404 (+ 271us) server_negotiation.cc:299] Negotiation successful
0926 11:40:24.041587 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":103,"server-negotiator.queue_time_us":42}
I20260926 11:40:24.041745 8188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:24.032877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:36643 (local address 127.0.0.1:57204)
0926 11:40:24.033175 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:24.033189 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:24.033302 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:24.033527 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:24.033529 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:24.033918 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:24.034016 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:24.034022 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:24.035121 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:24.035125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:24.035723 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:24.035733 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:24.036248 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:24.036250 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:24.036345 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:24.036894 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:24.036906 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:24.038485 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:24.040062 (+ 1577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:24.040072 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:24.040078 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:24.040363 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:24.040644 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:24.040647 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:24.040649 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:24.040734 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:24.041106 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:24.041113 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:24.041240 (+ 127us) client_negotiation.cc:769] Sending connection context
0926 11:40:24.041366 (+ 126us) client_negotiation.cc:240] Negotiation successful
0926 11:40:24.041537 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":16,"thread_start_us":123,"threads_started":1}
I20260926 11:40:24.046015 7642 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:57204:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 11:40:24.046787 7642 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:40:24.052296 7921 tablet_service.cc:1467] Processing CreateTablet for tablet ab2ced3731d54546887bb084f54b1aab (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5320e4f774d44b7e9e1b32b25239a424]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 11:40:24.052493 8061 tablet_service.cc:1467] Processing CreateTablet for tablet ab2ced3731d54546887bb084f54b1aab (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5320e4f774d44b7e9e1b32b25239a424]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 11:40:24.052724 7921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab2ced3731d54546887bb084f54b1aab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:24.052876 8061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab2ced3731d54546887bb084f54b1aab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:24.055214 8152 tablet_bootstrap.cc:492] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232: Bootstrap starting.
I20260926 11:40:24.055917 7781 tablet_service.cc:1467] Processing CreateTablet for tablet ab2ced3731d54546887bb084f54b1aab (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5320e4f774d44b7e9e1b32b25239a424]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 11:40:24.056083 8152 tablet_bootstrap.cc:654] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:24.056262 7781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab2ced3731d54546887bb084f54b1aab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:24.057433 8152 tablet_bootstrap.cc:492] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232: No bootstrap required, opened a new log
I20260926 11:40:24.057518 8152 ts_tablet_manager.cc:1397] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 11:40:24.057938 8152 raft_consensus.cc:348] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:24.058049 8152 raft_consensus.cc:374] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:24.058099 8152 raft_consensus.cc:729] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f67d214e4b6f4a3a88ba0810ec332232, State: Initialized, Role: FOLLOWER
I20260926 11:40:24.058219 8152 consensus_queue.cc:260] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:24.058454 8152 ts_tablet_manager.cc:1428] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:24.058636 8160 tablet_replica.cc:439] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } } }
I20260926 11:40:24.058761 8160 tablet_replica.cc:442] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 11:40:24.059275 8151 tablet_bootstrap.cc:492] T ab2ced3731d54546887bb084f54b1aab P 0ef370ad8a444940909f329391e4a540: Bootstrap starting.
I20260926 11:40:24.060176 8151 tablet_bootstrap.cc:654] T ab2ced3731d54546887bb084f54b1aab P 0ef370ad8a444940909f329391e4a540: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:24.061110 8150 tablet_bootstrap.cc:492] T ab2ced3731d54546887bb084f54b1aab P d316b542b21d424a84a494d4218f039b: Bootstrap starting.
I20260926 11:40:24.061297 8151 tablet_bootstrap.cc:492] T ab2ced3731d54546887bb084f54b1aab P 0ef370ad8a444940909f329391e4a540: No bootstrap required, opened a new log
I20260926 11:40:24.061372 8151 ts_tablet_manager.cc:1397] T ab2ced3731d54546887bb084f54b1aab P 0ef370ad8a444940909f329391e4a540: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:24.061815 8151 raft_consensus.cc:348] T ab2ced3731d54546887bb084f54b1aab P 0ef370ad8a444940909f329391e4a540 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:24.061923 8151 raft_consensus.cc:374] T ab2ced3731d54546887bb084f54b1aab P 0ef370ad8a444940909f329391e4a540 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:24.061954 8151 raft_consensus.cc:729] T ab2ced3731d54546887bb084f54b1aab P 0ef370ad8a444940909f329391e4a540 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ef370ad8a444940909f329391e4a540, State: Initialized, Role: FOLLOWER
I20260926 11:40:24.062039 8150 tablet_bootstrap.cc:654] T ab2ced3731d54546887bb084f54b1aab P d316b542b21d424a84a494d4218f039b: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:24.062076 8151 consensus_queue.cc:260] T ab2ced3731d54546887bb084f54b1aab P 0ef370ad8a444940909f329391e4a540 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:24.062356 8151 ts_tablet_manager.cc:1428] T ab2ced3731d54546887bb084f54b1aab P 0ef370ad8a444940909f329391e4a540: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:24.062446 8157 tablet_replica.cc:439] T ab2ced3731d54546887bb084f54b1aab P 0ef370ad8a444940909f329391e4a540: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } } }
I20260926 11:40:24.062579 8157 tablet_replica.cc:442] T ab2ced3731d54546887bb084f54b1aab P 0ef370ad8a444940909f329391e4a540: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 11:40:24.063530 8150 tablet_bootstrap.cc:492] T ab2ced3731d54546887bb084f54b1aab P d316b542b21d424a84a494d4218f039b: No bootstrap required, opened a new log
I20260926 11:40:24.063616 8150 ts_tablet_manager.cc:1397] T ab2ced3731d54546887bb084f54b1aab P d316b542b21d424a84a494d4218f039b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 11:40:24.064011 8150 raft_consensus.cc:348] T ab2ced3731d54546887bb084f54b1aab P d316b542b21d424a84a494d4218f039b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:24.064124 8150 raft_consensus.cc:374] T ab2ced3731d54546887bb084f54b1aab P d316b542b21d424a84a494d4218f039b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:24.064162 8150 raft_consensus.cc:729] T ab2ced3731d54546887bb084f54b1aab P d316b542b21d424a84a494d4218f039b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d316b542b21d424a84a494d4218f039b, State: Initialized, Role: FOLLOWER
I20260926 11:40:24.064283 8150 consensus_queue.cc:260] T ab2ced3731d54546887bb084f54b1aab P d316b542b21d424a84a494d4218f039b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:24.064519 8150 ts_tablet_manager.cc:1428] T ab2ced3731d54546887bb084f54b1aab P d316b542b21d424a84a494d4218f039b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 11:40:24.064502 8156 tablet_replica.cc:439] T ab2ced3731d54546887bb084f54b1aab P d316b542b21d424a84a494d4218f039b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } } }
I20260926 11:40:24.064670 8156 tablet_replica.cc:442] T ab2ced3731d54546887bb084f54b1aab P d316b542b21d424a84a494d4218f039b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 11:40:24.189725 8160 raft_consensus.cc:482] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:40:24.189867 8160 raft_consensus.cc:504] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:24.190330 8160 leader_election.cc:288] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d316b542b21d424a84a494d4218f039b (127.6.96.1:44457), 0ef370ad8a444940909f329391e4a540 (127.6.96.2:38389)
I20260926 11:40:24.190778 7941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab2ced3731d54546887bb084f54b1aab" candidate_uuid: "f67d214e4b6f4a3a88ba0810ec332232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef370ad8a444940909f329391e4a540" is_pre_election: true
I20260926 11:40:24.190781 7801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab2ced3731d54546887bb084f54b1aab" candidate_uuid: "f67d214e4b6f4a3a88ba0810ec332232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d316b542b21d424a84a494d4218f039b" is_pre_election: true
I20260926 11:40:24.190934 7941 raft_consensus.cc:2454] T ab2ced3731d54546887bb084f54b1aab P 0ef370ad8a444940909f329391e4a540 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f67d214e4b6f4a3a88ba0810ec332232 in term 0.
I20260926 11:40:24.190963 7801 raft_consensus.cc:2454] T ab2ced3731d54546887bb084f54b1aab P d316b542b21d424a84a494d4218f039b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f67d214e4b6f4a3a88ba0810ec332232 in term 0.
I20260926 11:40:24.191210 8014 leader_election.cc:302] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ef370ad8a444940909f329391e4a540, f67d214e4b6f4a3a88ba0810ec332232; no voters:
I20260926 11:40:24.191375 8160 raft_consensus.cc:2781] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:40:24.191433 8160 raft_consensus.cc:482] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:40:24.191478 8160 raft_consensus.cc:3037] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:24.192421 8160 raft_consensus.cc:504] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:24.192725 8160 leader_election.cc:288] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232 [CANDIDATE]: Term 1 election: Requested vote from peers d316b542b21d424a84a494d4218f039b (127.6.96.1:44457), 0ef370ad8a444940909f329391e4a540 (127.6.96.2:38389)
I20260926 11:40:24.193041 7801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab2ced3731d54546887bb084f54b1aab" candidate_uuid: "f67d214e4b6f4a3a88ba0810ec332232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d316b542b21d424a84a494d4218f039b"
I20260926 11:40:24.193063 7941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab2ced3731d54546887bb084f54b1aab" candidate_uuid: "f67d214e4b6f4a3a88ba0810ec332232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef370ad8a444940909f329391e4a540"
I20260926 11:40:24.193141 7801 raft_consensus.cc:3037] T ab2ced3731d54546887bb084f54b1aab P d316b542b21d424a84a494d4218f039b [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:24.193193 7941 raft_consensus.cc:3037] T ab2ced3731d54546887bb084f54b1aab P 0ef370ad8a444940909f329391e4a540 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:24.194020 7801 raft_consensus.cc:2454] T ab2ced3731d54546887bb084f54b1aab P d316b542b21d424a84a494d4218f039b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f67d214e4b6f4a3a88ba0810ec332232 in term 1.
I20260926 11:40:24.194046 7941 raft_consensus.cc:2454] T ab2ced3731d54546887bb084f54b1aab P 0ef370ad8a444940909f329391e4a540 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f67d214e4b6f4a3a88ba0810ec332232 in term 1.
I20260926 11:40:24.194252 8013 leader_election.cc:302] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d316b542b21d424a84a494d4218f039b, f67d214e4b6f4a3a88ba0810ec332232; no voters:
I20260926 11:40:24.194401 8160 raft_consensus.cc:2781] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:24.194480 8160 raft_consensus.cc:686] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232 [term 1 LEADER]: Becoming Leader. State: Replica: f67d214e4b6f4a3a88ba0810ec332232, State: Running, Role: LEADER
I20260926 11:40:24.194612 8160 consensus_queue.cc:237] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } }
I20260926 11:40:24.195276 8170 tablet_replica.cc:439] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232: TxnStatusTablet state changed. Reason: New leader f67d214e4b6f4a3a88ba0810ec332232. Latest consensus state: current_term: 1 leader_uuid: "f67d214e4b6f4a3a88ba0810ec332232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } } }
I20260926 11:40:24.195360 8170 tablet_replica.cc:442] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232: This TxnStatusTablet replica's current role is: LEADER
I20260926 11:40:24.195624 8193 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 11:40:24.195816 8193 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 11:40:24.196364 7644 catalog_manager.cc:5495] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232 reported cstate change: term changed from 0 to 1, leader changed from <none> to f67d214e4b6f4a3a88ba0810ec332232 (127.6.96.3). New cstate: current_term: 1 leader_uuid: "f67d214e4b6f4a3a88ba0810ec332232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } health_report { overall_health: HEALTHY } } }
I20260926 11:40:24.215668 8149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:24.212540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36312 (local address 127.6.96.3:36519)
0926 11:40:24.212656 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:24.212661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:24.212675 (+ 14us) server_negotiation.cc:407] Connection header received
0926 11:40:24.212859 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:24.212861 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:24.212908 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:24.212972 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:40:24.213224 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:24.213724 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:24.214548 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:24.214759 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:24.214874 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:24.215196 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:40:24.215296 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:40:24.215356 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:24.215476 (+ 120us) server_negotiation.cc:299] Negotiation successful
0926 11:40:24.215530 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 11:40:24.215668 8188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:24.212450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:36519 (local address 127.0.0.1:36312)
0926 11:40:24.212610 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:24.212625 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:24.212739 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:24.212995 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:24.212998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:24.213005 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 11:40:24.213104 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:24.213110 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:24.213827 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:24.213830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:24.214439 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:24.214447 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:24.214875 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:24.214878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:24.214973 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:24.215026 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 11:40:24.215374 (+ 348us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 11:40:24.215391 (+ 17us) client_negotiation.cc:769] Sending connection context
0926 11:40:24.215447 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 11:40:24.215532 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260926 11:40:24.218495 7801 raft_consensus.cc:1261] T ab2ced3731d54546887bb084f54b1aab P d316b542b21d424a84a494d4218f039b [term 1 FOLLOWER]: Refusing update from remote peer f67d214e4b6f4a3a88ba0810ec332232: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:24.218533 7941 raft_consensus.cc:1261] T ab2ced3731d54546887bb084f54b1aab P 0ef370ad8a444940909f329391e4a540 [term 1 FOLLOWER]: Refusing update from remote peer f67d214e4b6f4a3a88ba0810ec332232: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:24.218943 8170 consensus_queue.cc:1035] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:24.219221 8160 consensus_queue.cc:1035] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:24.220477 8157 tablet_replica.cc:439] T ab2ced3731d54546887bb084f54b1aab P 0ef370ad8a444940909f329391e4a540: TxnStatusTablet state changed. Reason: New leader f67d214e4b6f4a3a88ba0810ec332232. Latest consensus state: current_term: 1 leader_uuid: "f67d214e4b6f4a3a88ba0810ec332232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } } }
I20260926 11:40:24.220587 8157 tablet_replica.cc:442] T ab2ced3731d54546887bb084f54b1aab P 0ef370ad8a444940909f329391e4a540: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 11:40:24.220665 8156 tablet_replica.cc:439] T ab2ced3731d54546887bb084f54b1aab P d316b542b21d424a84a494d4218f039b: TxnStatusTablet state changed. Reason: New leader f67d214e4b6f4a3a88ba0810ec332232. Latest consensus state: current_term: 1 leader_uuid: "f67d214e4b6f4a3a88ba0810ec332232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } } }
I20260926 11:40:24.220780 8156 tablet_replica.cc:442] T ab2ced3731d54546887bb084f54b1aab P d316b542b21d424a84a494d4218f039b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 11:40:24.222174 8170 tablet_replica.cc:439] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f67d214e4b6f4a3a88ba0810ec332232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } } }
I20260926 11:40:24.222292 8170 tablet_replica.cc:442] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232: This TxnStatusTablet replica's current role is: LEADER
I20260926 11:40:24.222391 8163 mvcc.cc:205] Tried to move back new op lower bound from 7333571887994691584 to 7333571887902355456. Current Snapshot: MvccSnapshot[applied={T|T < 7333571887994691584 or (T in {7333571887994691584})}]
I20260926 11:40:24.223148 8156 tablet_replica.cc:439] T ab2ced3731d54546887bb084f54b1aab P d316b542b21d424a84a494d4218f039b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f67d214e4b6f4a3a88ba0810ec332232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } } }
I20260926 11:40:24.223259 8156 tablet_replica.cc:442] T ab2ced3731d54546887bb084f54b1aab P d316b542b21d424a84a494d4218f039b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 11:40:24.223766 8170 tablet_replica.cc:439] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f67d214e4b6f4a3a88ba0810ec332232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } } }
I20260926 11:40:24.223862 8170 tablet_replica.cc:442] T ab2ced3731d54546887bb084f54b1aab P f67d214e4b6f4a3a88ba0810ec332232: This TxnStatusTablet replica's current role is: LEADER
I20260926 11:40:24.225138 8165 mvcc.cc:205] Tried to move back new op lower bound from 7333571887994691584 to 7333571887902355456. Current Snapshot: MvccSnapshot[applied={T|T < 7333571887994691584}]
I20260926 11:40:24.226123 8157 tablet_replica.cc:439] T ab2ced3731d54546887bb084f54b1aab P 0ef370ad8a444940909f329391e4a540: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f67d214e4b6f4a3a88ba0810ec332232" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d316b542b21d424a84a494d4218f039b" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44457 } } peers { permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 } } peers { permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 } } }
I20260926 11:40:24.226234 8157 tablet_replica.cc:442] T ab2ced3731d54546887bb084f54b1aab P 0ef370ad8a444940909f329391e4a540: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 11:40:24.231756 8148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:24.228203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59216 (local address 127.6.96.1:44457)
0926 11:40:24.228315 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:24.228319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:24.228367 (+ 48us) server_negotiation.cc:407] Connection header received
0926 11:40:24.228514 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:24.228517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:24.228570 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:24.228656 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:40:24.228867 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:24.229380 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:24.230154 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:24.230431 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:24.230516 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:24.230804 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:40:24.231305 (+ 501us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:40:24.231408 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:24.231547 (+ 139us) server_negotiation.cc:299] Negotiation successful
0926 11:40:24.231625 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 11:40:24.241909 8149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:24.238805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:52037 (local address 127.6.96.3:36519)
0926 11:40:24.238952 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:24.238957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:24.239086 (+ 129us) server_negotiation.cc:407] Connection header received
0926 11:40:24.239202 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:24.239205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:24.239261 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:24.239319 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:40:24.239557 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:24.240052 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:24.240904 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:24.241110 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:24.241191 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:24.241496 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:40:24.241566 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:40:24.241619 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:24.241706 (+ 87us) server_negotiation.cc:299] Negotiation successful
0926 11:40:24.241762 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 11:40:24.242134 8200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:24.238684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:36519 (local address 127.6.96.1:52037)
0926 11:40:24.239006 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:24.239019 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:24.239119 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:24.239333 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:24.239336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:24.239343 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 11:40:24.239460 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:24.239466 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:24.240153 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:24.240156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:24.240792 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:24.240800 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:24.241203 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:24.241206 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:24.241325 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:24.241369 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 11:40:24.241634 (+ 265us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 11:40:24.241641 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 11:40:24.241697 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 11:40:24.241775 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":173,"threads_started":1}
I20260926 11:40:24.249840 7941 raft_consensus.cc:1261] T 3df1522c79b649a5820960dc4bfea031 P 0ef370ad8a444940909f329391e4a540 [term 1 FOLLOWER]: Refusing update from remote peer d316b542b21d424a84a494d4218f039b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:24.249981 8081 raft_consensus.cc:1261] T 3df1522c79b649a5820960dc4bfea031 P f67d214e4b6f4a3a88ba0810ec332232 [term 1 FOLLOWER]: Refusing update from remote peer d316b542b21d424a84a494d4218f039b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:24.250317 8156 consensus_queue.cc:1035] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ef370ad8a444940909f329391e4a540" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 38389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:24.250496 8181 consensus_queue.cc:1035] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f67d214e4b6f4a3a88ba0810ec332232" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:24.254407 8164 mvcc.cc:205] Tried to move back new op lower bound from 7333571888120750080 to 7333571887074709504. Current Snapshot: MvccSnapshot[applied={T|T < 7333571888120750080}]
I20260926 11:40:24.273237 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:24.269601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:44457 (local address 127.6.96.3:53435)
0926 11:40:24.269874 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:24.269890 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:24.269979 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:24.270192 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:24.270193 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:24.270198 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 11:40:24.270298 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:24.270302 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:24.271200 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:24.271203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:24.272028 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:24.272037 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:24.272579 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:24.272581 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:24.272668 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:24.272743 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 11:40:24.272996 (+ 253us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 11:40:24.273001 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 11:40:24.273053 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 11:40:24.273118 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":74,"threads_started":1}
I20260926 11:40:24.273242 8148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:24.269848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:53435 (local address 127.6.96.1:44457)
0926 11:40:24.269970 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:24.269974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:24.269990 (+ 16us) server_negotiation.cc:407] Connection header received
0926 11:40:24.270053 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:24.270056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:24.270108 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:24.270178 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:40:24.270374 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:24.271112 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:24.272135 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:24.272480 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:24.272584 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:24.272843 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:40:24.272922 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:40:24.272989 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:24.273063 (+ 74us) server_negotiation.cc:299] Negotiation successful
0926 11:40:24.273115 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 11:40:24.301457 7642 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:24.301636 7642 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:24.303877 7642 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 900e2cd538cf416c8a7a4f1fbb707cb8: Sending DeleteTablet for 3 replicas of tablet 3df1522c79b649a5820960dc4bfea031
I20260926 11:40:24.304576 7921 tablet_service.cc:1514] Processing DeleteTablet for tablet 3df1522c79b649a5820960dc4bfea031 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:24 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:56866
I20260926 11:40:24.304768 8174 tablet_replica.cc:330] T 3df1522c79b649a5820960dc4bfea031 P 0ef370ad8a444940909f329391e4a540: stopping tablet replica
I20260926 11:40:24.304764 7781 tablet_service.cc:1514] Processing DeleteTablet for tablet 3df1522c79b649a5820960dc4bfea031 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:24 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:59206
I20260926 11:40:24.304880 8174 raft_consensus.cc:2229] T 3df1522c79b649a5820960dc4bfea031 P 0ef370ad8a444940909f329391e4a540 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:24.304943 8175 tablet_replica.cc:330] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b: stopping tablet replica
I20260926 11:40:24.304989 8174 raft_consensus.cc:2258] T 3df1522c79b649a5820960dc4bfea031 P 0ef370ad8a444940909f329391e4a540 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:24.305044 8175 raft_consensus.cc:2229] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b [term 1 LEADER]: Raft consensus shutting down.
I20260926 11:40:24.305380 8175 raft_consensus.cc:2258] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:24.305748 8061 tablet_service.cc:1514] Processing DeleteTablet for tablet 3df1522c79b649a5820960dc4bfea031 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:24 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:36292
I20260926 11:40:24.305932 8173 tablet_replica.cc:330] T 3df1522c79b649a5820960dc4bfea031 P f67d214e4b6f4a3a88ba0810ec332232: stopping tablet replica
I20260926 11:40:24.306047 8173 raft_consensus.cc:2229] T 3df1522c79b649a5820960dc4bfea031 P f67d214e4b6f4a3a88ba0810ec332232 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:24.306159 8173 raft_consensus.cc:2258] T 3df1522c79b649a5820960dc4bfea031 P f67d214e4b6f4a3a88ba0810ec332232 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:24.306977 8175 ts_tablet_manager.cc:1905] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:24.309892 8175 ts_tablet_manager.cc:1918] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 11:40:24.309973 8175 log.cc:1196] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-0/wal/wals/3df1522c79b649a5820960dc4bfea031
I20260926 11:40:24.310209 8175 ts_tablet_manager.cc:1939] T 3df1522c79b649a5820960dc4bfea031 P d316b542b21d424a84a494d4218f039b: Deleting consensus metadata
I20260926 11:40:24.311038 7629 catalog_manager.cc:4841] TS d316b542b21d424a84a494d4218f039b (127.6.96.1:44457): tablet 3df1522c79b649a5820960dc4bfea031 (table test-table [id=a9e196508749469a873319e76c215f38]) successfully deleted
I20260926 11:40:24.311295 8173 ts_tablet_manager.cc:1905] T 3df1522c79b649a5820960dc4bfea031 P f67d214e4b6f4a3a88ba0810ec332232: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:24.314002 8173 ts_tablet_manager.cc:1918] T 3df1522c79b649a5820960dc4bfea031 P f67d214e4b6f4a3a88ba0810ec332232: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 11:40:24.314083 8173 log.cc:1196] T 3df1522c79b649a5820960dc4bfea031 P f67d214e4b6f4a3a88ba0810ec332232: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/3df1522c79b649a5820960dc4bfea031
I20260926 11:40:24.314320 8173 ts_tablet_manager.cc:1939] T 3df1522c79b649a5820960dc4bfea031 P f67d214e4b6f4a3a88ba0810ec332232: Deleting consensus metadata
I20260926 11:40:24.315126 8174 ts_tablet_manager.cc:1905] T 3df1522c79b649a5820960dc4bfea031 P 0ef370ad8a444940909f329391e4a540: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:24.316362 7629 catalog_manager.cc:4841] TS f67d214e4b6f4a3a88ba0810ec332232 (127.6.96.3:36519): tablet 3df1522c79b649a5820960dc4bfea031 (table test-table [id=a9e196508749469a873319e76c215f38]) successfully deleted
I20260926 11:40:24.318059 8174 ts_tablet_manager.cc:1918] T 3df1522c79b649a5820960dc4bfea031 P 0ef370ad8a444940909f329391e4a540: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 11:40:24.318130 8174 log.cc:1196] T 3df1522c79b649a5820960dc4bfea031 P 0ef370ad8a444940909f329391e4a540: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790422817500949-6528-0/minicluster-data/ts-1/wal/wals/3df1522c79b649a5820960dc4bfea031
I20260926 11:40:24.318413 8174 ts_tablet_manager.cc:1939] T 3df1522c79b649a5820960dc4bfea031 P 0ef370ad8a444940909f329391e4a540: Deleting consensus metadata
I20260926 11:40:24.319180 7630 catalog_manager.cc:4841] TS 0ef370ad8a444940909f329391e4a540 (127.6.96.2:38389): tablet 3df1522c79b649a5820960dc4bfea031 (table test-table [id=a9e196508749469a873319e76c215f38]) successfully deleted
Sep 26 11:40:24 dist-test-slave-xxxx krb5kdc[7599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422823, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.96.1@KRBTEST.COM
I20260926 11:40:24.331156 8148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:24.316912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59232 (local address 127.6.96.1:44457)
0926 11:40:24.318998 (+ 2086us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:24.319003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:24.319433 (+ 430us) server_negotiation.cc:407] Connection header received
0926 11:40:24.319616 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:24.319620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:24.319676 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:24.319764 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:24.320573 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:24.321366 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:24.322620 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:24.322976 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:24.323089 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:24.326703 (+ 3614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:24.326715 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:24.326731 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:24.326761 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:24.328734 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:24.329572 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:24.329579 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:24.329580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:24.329673 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:24.329952 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:24.329955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:24.329957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:24.330278 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:24.330443 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:24.330855 (+ 412us) server_negotiation.cc:299] Negotiation successful
0926 11:40:24.330985 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1994}
W20260926 11:40:24.332096 7841 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59232
I20260926 11:40:24.347299 7693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:24.337752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57218 (local address 127.6.96.62:36643)
0926 11:40:24.337874 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:24.337879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:24.338334 (+ 455us) server_negotiation.cc:407] Connection header received
0926 11:40:24.338557 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:24.338561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:24.338612 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:24.338662 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:24.339612 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:24.340452 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:24.341719 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:24.342048 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:24.342119 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:24.343902 (+ 1783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:24.343907 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:24.343909 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:24.343938 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:24.345500 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:24.345935 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:24.345939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:24.345941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:24.346018 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:24.346298 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:24.346301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:24.346303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:24.346502 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:24.346596 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:24.347010 (+ 414us) server_negotiation.cc:299] Negotiation successful
0926 11:40:24.347122 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260926 11:40:24.348176 7642 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57218
I20260926 11:40:24.349651 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 7716
I20260926 11:40:24.355789 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 7856
I20260926 11:40:24.360548 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 7996
I20260926 11:40:24.366106 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 7612
2026-09-26T11:40:24Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2535 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:40:24 dist-test-slave-xxxx krb5kdc[8214](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:40:24 dist-test-slave-xxxx krb5kdc[8214](info): set up 2 sockets
Sep 26 11:40:24 dist-test-slave-xxxx krb5kdc[8214](info): commencing operation
krb5kdc: starting...
W20260926 11:40:25.322410 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:40:25 dist-test-slave-xxxx krb5kdc[8214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:40:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:40:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:40:25.437067 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:38135
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:35363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:38135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:25.546020 8227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:25.546312 8227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:25.546380 8227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:25.550418 8227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:40:25.550477 8227 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 11:40:25.550498 8227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:25.550523 8227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:25.550544 8227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:40:25.550570 8227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:40:25.555821 8227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:35363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:38135
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:38135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:25.556880 8227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:25.557883 8227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:25.565861 8232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:25.565862 8233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:25.565862 8235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:25.565898 8227 server_base.cc:1034] running on GCE node
I20260926 11:40:25.566547 8227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:25.567653 8227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:25.568830 8227 hybrid_clock.cc:648] HybridClock initialized: now 1790422825568808 us; error 34 us; skew 500 ppm
Sep 26 11:40:25 dist-test-slave-xxxx krb5kdc[8214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422825, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:25.571465 8227 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:40:25.572336 8227 webserver.cc:460] Webserver started at http://127.6.96.62:35759/ using document root <none> and password file <none>
I20260926 11:40:25.572871 8227 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:25.572937 8227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:25.573127 8227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:25.575258 8227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "a0b52746549b43fe83890e93abf22be8"
format_stamp: "Formatted at 2026-09-26 11:40:25 on dist-test-slave-xxxx"
server_key: "6ee4b1863f03848f6c72dffcb1802bf7"
server_key_iv: "3a4da002e875dcb5f7828ed49052f682"
server_key_version: "encryptionkey@0"
I20260926 11:40:25.575824 8227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "a0b52746549b43fe83890e93abf22be8"
format_stamp: "Formatted at 2026-09-26 11:40:25 on dist-test-slave-xxxx"
server_key: "6ee4b1863f03848f6c72dffcb1802bf7"
server_key_iv: "3a4da002e875dcb5f7828ed49052f682"
server_key_version: "encryptionkey@0"
I20260926 11:40:25.579665 8227 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 11:40:25.582291 8242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:25.583652 8227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:40:25.583779 8227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "a0b52746549b43fe83890e93abf22be8"
format_stamp: "Formatted at 2026-09-26 11:40:25 on dist-test-slave-xxxx"
server_key: "6ee4b1863f03848f6c72dffcb1802bf7"
server_key_iv: "3a4da002e875dcb5f7828ed49052f682"
server_key_version: "encryptionkey@0"
I20260926 11:40:25.583894 8227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:25.610651 8227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:25.613688 8227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:25.613873 8227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:25.621295 8227 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:38135
I20260926 11:40:25.621351 8304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:38135 every 8 connection(s)
I20260926 11:40:25.622234 8227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:40:25.622977 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 8227
I20260926 11:40:25.623095 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:40:25.623414 6528 external_mini_cluster.cc:1428] Setting key 44ce9bac1529aea54658f5d69baa01dd
I20260926 11:40:25.625623 8305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:25.631464 8305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8: Bootstrap starting.
Sep 26 11:40:25 dist-test-slave-xxxx krb5kdc[8214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:25.634037 8305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:25.635304 8305 log.cc:824] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:25.637418 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:25.624763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47218 (local address 127.6.96.62:38135)
0926 11:40:25.625173 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:25.625187 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:25.625222 (+ 35us) server_negotiation.cc:407] Connection header received
0926 11:40:25.625875 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:25.625891 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:25.626215 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:25.626525 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:25.627305 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:25.628315 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:25.629303 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:25.629999 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:25.630175 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:25.632502 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:25.632509 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:25.632527 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:25.632560 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:25.634385 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:25.634767 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:25.634773 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:25.634779 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:25.634941 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:25.635186 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:25.635190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:25.635191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:25.635691 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:25.635845 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:25.636211 (+ 366us) server_negotiation.cc:299] Negotiation successful
0926 11:40:25.636501 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":127,"threads_started":1}
I20260926 11:40:25.637526 8305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8: No bootstrap required, opened a new log
I20260926 11:40:25.640724 8305 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b52746549b43fe83890e93abf22be8" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 38135 } }
I20260926 11:40:25.640991 8305 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:25.641049 8305 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0b52746549b43fe83890e93abf22be8, State: Initialized, Role: FOLLOWER
I20260926 11:40:25.641456 8305 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b52746549b43fe83890e93abf22be8" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 38135 } }
I20260926 11:40:25.641571 8305 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:40:25.641628 8305 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:40:25.641721 8305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:25.642525 8305 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b52746549b43fe83890e93abf22be8" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 38135 } }
I20260926 11:40:25.642851 8305 leader_election.cc:302] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0b52746549b43fe83890e93abf22be8; no voters:
I20260926 11:40:25.643136 8305 leader_election.cc:288] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:40:25.643263 8310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:25.643476 8310 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8 [term 1 LEADER]: Becoming Leader. State: Replica: a0b52746549b43fe83890e93abf22be8, State: Running, Role: LEADER
I20260926 11:40:25.643743 8310 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b52746549b43fe83890e93abf22be8" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 38135 } }
I20260926 11:40:25.644222 8305 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:40:25.645220 8311 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0b52746549b43fe83890e93abf22be8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b52746549b43fe83890e93abf22be8" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 38135 } } }
I20260926 11:40:25.645345 8311 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:25.645562 8312 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0b52746549b43fe83890e93abf22be8. Latest consensus state: current_term: 1 leader_uuid: "a0b52746549b43fe83890e93abf22be8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b52746549b43fe83890e93abf22be8" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 38135 } } }
I20260926 11:40:25.645645 8312 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:25.645769 8319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:40:25.648805 8319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:40:25.654641 8319 catalog_manager.cc:1347] Generated new cluster ID: daca43c8746943ecbd7f05b5233e1ef3
I20260926 11:40:25.654738 8319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:40:25.668435 8319 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:40:25.669431 8319 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:40:25.674187 8319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8: Generated new TSK 0
I20260926 11:40:25.675104 8319 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.96.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
I20260926 11:40:25.713904 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:38135
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:35363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:25.838881 8331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:25.839150 8331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:25.839197 8331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:25.843293 8331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:25.843359 8331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:25.843400 8331 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 11:40:25.843411 8331 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 11:40:25.843453 8331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:40:25.848766 8331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:35363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.96.62:38135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:25.849889 8331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:25.850963 8331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:25.860168 8336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:25.860263 8339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:25.860335 8337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:25.860390 8331 server_base.cc:1034] running on GCE node
I20260926 11:40:25.860810 8331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:25.861435 8331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:25.862596 8331 hybrid_clock.cc:648] HybridClock initialized: now 1790422825862578 us; error 34 us; skew 500 ppm
Sep 26 11:40:25 dist-test-slave-xxxx krb5kdc[8214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422825, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:25.865232 8331 init.cc:377] Logged in from keytab as kudu/127.6.96.1@KRBTEST.COM (short username kudu)
I20260926 11:40:25.866050 8331 webserver.cc:460] Webserver started at http://127.6.96.1:32921/ using document root <none> and password file <none>
I20260926 11:40:25.866560 8331 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:25.866623 8331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:25.866848 8331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:25.868803 8331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "0a06f0913f7745efa75ccfde9eaf9c8d"
format_stamp: "Formatted at 2026-09-26 11:40:25 on dist-test-slave-xxxx"
server_key: "580e81e41446ab9b6ee4a1ec9520c683"
server_key_iv: "a5d9330fae5231ae24a40c2617950f3d"
server_key_version: "encryptionkey@0"
I20260926 11:40:25.869335 8331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "0a06f0913f7745efa75ccfde9eaf9c8d"
format_stamp: "Formatted at 2026-09-26 11:40:25 on dist-test-slave-xxxx"
server_key: "580e81e41446ab9b6ee4a1ec9520c683"
server_key_iv: "a5d9330fae5231ae24a40c2617950f3d"
server_key_version: "encryptionkey@0"
I20260926 11:40:25.872880 8331 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 11:40:25.875314 8346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:25.876505 8331 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:40:25.876606 8331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "0a06f0913f7745efa75ccfde9eaf9c8d"
format_stamp: "Formatted at 2026-09-26 11:40:25 on dist-test-slave-xxxx"
server_key: "580e81e41446ab9b6ee4a1ec9520c683"
server_key_iv: "a5d9330fae5231ae24a40c2617950f3d"
server_key_version: "encryptionkey@0"
I20260926 11:40:25.876715 8331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:25.898675 8331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:25.901736 8331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:25.901921 8331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:25.903461 8331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:25.903512 8331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:25.903565 8331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:25.903594 8331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 11:40:25 dist-test-slave-xxxx krb5kdc[8214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422825, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:25.913298 8331 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:34477
I20260926 11:40:25.913337 8461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:34477 every 8 connection(s)
I20260926 11:40:25.914260 8331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:40:25.917064 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:25.904189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:52797 (local address 127.6.96.62:38135)
0926 11:40:25.904302 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:25.904305 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:25.905376 (+ 1071us) server_negotiation.cc:407] Connection header received
0926 11:40:25.906379 (+ 1003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:25.906383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:25.906440 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:25.906516 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:25.908028 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:25.908866 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:25.910554 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:25.910954 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:25.911061 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:25.913882 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:25.913886 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:25.913889 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:25.913918 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:25.915472 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:25.915896 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:25.915899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:25.915900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:25.915971 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:25.916200 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:25.916202 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:25.916203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:25.916367 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:25.916438 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:25.916845 (+ 407us) server_negotiation.cc:299] Negotiation successful
0926 11:40:25.916944 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 11:40:25.917863 8355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:25.904759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:38135 (local address 127.6.96.1:52797)
0926 11:40:25.905232 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:25.905273 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:25.906164 (+ 891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:25.906750 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:25.906760 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:25.907227 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:25.907871 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:25.907893 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:25.909016 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:25.909019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:25.910433 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:25.910444 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:25.911114 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:25.911118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:25.911299 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:25.912008 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:25.912033 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:25.913712 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:25.915566 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:25.915571 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:25.915581 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:25.915821 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:25.916025 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:25.916027 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:25.916028 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:25.916137 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:25.916439 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:25.916443 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:25.916661 (+ 218us) client_negotiation.cc:769] Sending connection context
0926 11:40:25.916813 (+ 152us) client_negotiation.cc:240] Negotiation successful
0926 11:40:25.917050 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":139,"threads_started":1}
I20260926 11:40:25.919147 8462 heartbeater.cc:347] Connected to a master server at 127.6.96.62:38135
I20260926 11:40:25.919287 8462 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:25.919849 8462 heartbeater.cc:511] Master 127.6.96.62:38135 requested a full tablet report, sending...
I20260926 11:40:25.920708 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 8331
I20260926 11:40:25.920828 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:40:25.921110 6528 external_mini_cluster.cc:1428] Setting key 7224abce3e6c81b144ce8bc6bf0aeca9
I20260926 11:40:25.921392 8258 ts_manager.cc:194] Registered new tserver with Master: 0a06f0913f7745efa75ccfde9eaf9c8d (127.6.96.1:34477)
I20260926 11:40:25.922613 8258 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.1@KRBTEST.COM'} at 127.6.96.1:52797
WARNING: no policy specified for kudu/127.6.96.2@KRBTEST.COM; defaulting to no policy
I20260926 11:40:25.930297 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:25.923127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:32953 (local address 127.6.96.62:38135)
0926 11:40:25.923239 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:25.923242 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:25.923701 (+ 459us) server_negotiation.cc:407] Connection header received
0926 11:40:25.923874 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:25.923877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:25.923918 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:25.923984 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:25.924392 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:25.924892 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:25.925947 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:25.926160 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:25.926254 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:25.927507 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:25.927510 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:25.927512 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:25.927535 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:25.928817 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:25.929257 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:25.929260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:25.929261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:25.929331 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:25.929580 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:25.929582 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:25.929583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:25.929731 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:25.929798 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:25.930051 (+ 253us) server_negotiation.cc:299] Negotiation successful
0926 11:40:25.930141 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 11:40:25.930335 8468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:25.923273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:38135 (local address 127.6.96.1:32953)
0926 11:40:25.923621 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:25.923635 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:25.923771 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:25.924007 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:25.924009 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:25.924190 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:25.924292 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:25.924298 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:25.925006 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:25.925010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:25.925816 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:25.925826 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:25.926284 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:25.926289 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:25.926383 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:25.926886 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:25.926900 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:25.927323 (+ 423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:25.928922 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:25.928926 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:25.928928 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:25.929170 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:25.929410 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:25.929413 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:25.929415 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:25.929507 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:25.929813 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:25.929816 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:25.929910 (+ 94us) client_negotiation.cc:769] Sending connection context
0926 11:40:25.930005 (+ 95us) client_negotiation.cc:240] Negotiation successful
0926 11:40:25.930138 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":93,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
I20260926 11:40:25.945525 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:38135
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:35363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:26.050604 8471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:26.050915 8471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:26.050984 8471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:26.054981 8471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:26.055040 8471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:26.055094 8471 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 11:40:26.055122 8471 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 11:40:26.055171 8471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:40:26.060371 8471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:35363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.96.62:38135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:26.061347 8471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:26.062300 8471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:26.071316 8476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:26.071364 8471 server_base.cc:1034] running on GCE node
W20260926 11:40:26.071326 8479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:26.071368 8477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:26.071918 8471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:26.072525 8471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:26.073697 8471 hybrid_clock.cc:648] HybridClock initialized: now 1790422826073671 us; error 39 us; skew 500 ppm
Sep 26 11:40:26 dist-test-slave-xxxx krb5kdc[8214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422826, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:26.076280 8471 init.cc:377] Logged in from keytab as kudu/127.6.96.2@KRBTEST.COM (short username kudu)
I20260926 11:40:26.077111 8471 webserver.cc:460] Webserver started at http://127.6.96.2:37731/ using document root <none> and password file <none>
I20260926 11:40:26.077611 8471 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:26.077668 8471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:26.077826 8471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:26.079823 8471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "ea9ce406518f47f18242a8d987510090"
format_stamp: "Formatted at 2026-09-26 11:40:26 on dist-test-slave-xxxx"
server_key: "da5cea0a4130033eb63a560d77749275"
server_key_iv: "7559e318907b44775d326a02cef93233"
server_key_version: "encryptionkey@0"
I20260926 11:40:26.080323 8471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "ea9ce406518f47f18242a8d987510090"
format_stamp: "Formatted at 2026-09-26 11:40:26 on dist-test-slave-xxxx"
server_key: "da5cea0a4130033eb63a560d77749275"
server_key_iv: "7559e318907b44775d326a02cef93233"
server_key_version: "encryptionkey@0"
I20260926 11:40:26.083817 8471 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 11:40:26.086113 8486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:26.087407 8471 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:40:26.087512 8471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "ea9ce406518f47f18242a8d987510090"
format_stamp: "Formatted at 2026-09-26 11:40:26 on dist-test-slave-xxxx"
server_key: "da5cea0a4130033eb63a560d77749275"
server_key_iv: "7559e318907b44775d326a02cef93233"
server_key_version: "encryptionkey@0"
I20260926 11:40:26.087606 8471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:26.103926 8471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:26.106851 8471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:26.107025 8471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:26.108496 8471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:26.108551 8471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:26.108604 8471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:26.108632 8471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 11:40:26 dist-test-slave-xxxx krb5kdc[8214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422826, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:26.118651 8471 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:35251
I20260926 11:40:26.119155 8601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:35251 every 8 connection(s)
I20260926 11:40:26.119680 8471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:40:26.121122 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 8471
I20260926 11:40:26.121217 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:40:26.121464 6528 external_mini_cluster.cc:1428] Setting key f076c0206b1a29149c107c275d5eb85f
I20260926 11:40:26.122148 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.109273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:56765 (local address 127.6.96.62:38135)
0926 11:40:26.109412 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:26.109415 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:26.110449 (+ 1034us) server_negotiation.cc:407] Connection header received
0926 11:40:26.111562 (+ 1113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:26.111567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:26.111622 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:26.111683 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:26.113163 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.113716 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.114892 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.115114 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.115206 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.118098 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:26.118102 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:26.118105 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:26.118137 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:26.120047 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:26.120665 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:26.120669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:26.120670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:26.120758 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:26.121084 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:26.121088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:26.121090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:26.121292 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:26.121386 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:26.121872 (+ 486us) server_negotiation.cc:299] Negotiation successful
0926 11:40:26.121995 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 11:40:26.123001 8495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.109810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:38135 (local address 127.6.96.2:56765)
0926 11:40:26.110291 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:26.110335 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:26.111317 (+ 982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:26.112057 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:26.112063 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:26.112450 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:26.113022 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:26.113036 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.113851 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.113854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:26.114751 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:26.114760 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.115279 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.115283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:26.115478 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.116239 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:26.116255 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:26.117913 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:26.120191 (+ 2278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:26.120197 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:26.120212 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:26.120557 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:26.120840 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:26.120843 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:26.120845 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:26.120996 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:26.121388 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:26.121395 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:26.121656 (+ 261us) client_negotiation.cc:769] Sending connection context
0926 11:40:26.121817 (+ 161us) client_negotiation.cc:240] Negotiation successful
0926 11:40:26.122132 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":132,"threads_started":1}
I20260926 11:40:26.124207 8602 heartbeater.cc:347] Connected to a master server at 127.6.96.62:38135
I20260926 11:40:26.124368 8602 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:26.124855 8602 heartbeater.cc:511] Master 127.6.96.62:38135 requested a full tablet report, sending...
I20260926 11:40:26.125831 8259 ts_manager.cc:194] Registered new tserver with Master: ea9ce406518f47f18242a8d987510090 (127.6.96.2:35251)
I20260926 11:40:26.126518 8259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.2@KRBTEST.COM'} at 127.6.96.2:56765
WARNING: no policy specified for kudu/127.6.96.3@KRBTEST.COM; defaulting to no policy
I20260926 11:40:26.132828 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.126107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:56071 (local address 127.6.96.62:38135)
0926 11:40:26.126204 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:26.126207 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:26.126432 (+ 225us) server_negotiation.cc:407] Connection header received
0926 11:40:26.126558 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:26.126561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:26.126604 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:26.126675 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:26.127285 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.127776 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.128761 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.128968 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.129052 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.130051 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:26.130055 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:26.130057 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:26.130079 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:26.131327 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:26.131701 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:26.131704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:26.131706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:26.131773 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:26.132002 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:26.132005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:26.132006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:26.132244 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:26.132329 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:26.132578 (+ 249us) server_negotiation.cc:299] Negotiation successful
0926 11:40:26.132678 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 11:40:26.132838 8609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.126052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:38135 (local address 127.6.96.2:56071)
0926 11:40:26.126350 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:26.126364 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:26.126457 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:26.126696 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:26.126698 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:26.127046 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:26.127165 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:26.127173 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.127899 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.127902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:26.128666 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:26.128676 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.129072 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.129076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:26.129177 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.129623 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:26.129634 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:26.129931 (+ 297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:26.131427 (+ 1496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:26.131430 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:26.131433 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:26.131615 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:26.131849 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:26.131852 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:26.131854 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:26.131929 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:26.132333 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:26.132336 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:26.132434 (+ 98us) client_negotiation.cc:769] Sending connection context
0926 11:40:26.132528 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 11:40:26.132667 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":86,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
I20260926 11:40:26.149943 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:38135
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:35363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:26.258930 8611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:26.259173 8611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:26.259219 8611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:26.263113 8611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:26.263167 8611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:26.263206 8611 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 11:40:26.263218 8611 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 11:40:26.263248 8611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:40:26.268590 8611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:35363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.96.62:38135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:26.269572 8611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:26.270533 8611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:26.279940 8616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:26.279950 8617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:26.280004 8611 server_base.cc:1034] running on GCE node
W20260926 11:40:26.279934 8619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:26.280550 8611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:26.281213 8611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:26.282371 8611 hybrid_clock.cc:648] HybridClock initialized: now 1790422826282355 us; error 36 us; skew 500 ppm
Sep 26 11:40:26 dist-test-slave-xxxx krb5kdc[8214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422826, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:26.289129 8611 init.cc:377] Logged in from keytab as kudu/127.6.96.3@KRBTEST.COM (short username kudu)
I20260926 11:40:26.290095 8611 webserver.cc:460] Webserver started at http://127.6.96.3:45541/ using document root <none> and password file <none>
I20260926 11:40:26.290668 8611 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:26.290751 8611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:26.290973 8611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:26.293044 8611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "fa45b8b0831246c5b2fde85e78124b88"
format_stamp: "Formatted at 2026-09-26 11:40:26 on dist-test-slave-xxxx"
server_key: "cdbbea1a8fee79ec96e96520b09e53c5"
server_key_iv: "da09e3ac3e34dd56f0a5dbcc6d804c38"
server_key_version: "encryptionkey@0"
I20260926 11:40:26.293572 8611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "fa45b8b0831246c5b2fde85e78124b88"
format_stamp: "Formatted at 2026-09-26 11:40:26 on dist-test-slave-xxxx"
server_key: "cdbbea1a8fee79ec96e96520b09e53c5"
server_key_iv: "da09e3ac3e34dd56f0a5dbcc6d804c38"
server_key_version: "encryptionkey@0"
I20260926 11:40:26.297335 8611 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260926 11:40:26.299753 8626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:26.300964 8611 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:40:26.301062 8611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "fa45b8b0831246c5b2fde85e78124b88"
format_stamp: "Formatted at 2026-09-26 11:40:26 on dist-test-slave-xxxx"
server_key: "cdbbea1a8fee79ec96e96520b09e53c5"
server_key_iv: "da09e3ac3e34dd56f0a5dbcc6d804c38"
server_key_version: "encryptionkey@0"
I20260926 11:40:26.301172 8611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:26.310724 8611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:26.313349 8611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:26.313504 8611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:26.314812 8611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:26.314906 8611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:26.314949 8611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:26.314976 8611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 11:40:26 dist-test-slave-xxxx krb5kdc[8214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422826, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:26.324908 8611 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:36655
I20260926 11:40:26.324932 8741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:36655 every 8 connection(s)
I20260926 11:40:26.325940 8611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:40:26.328326 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.315615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:60325 (local address 127.6.96.62:38135)
0926 11:40:26.315746 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:26.315766 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:26.316556 (+ 790us) server_negotiation.cc:407] Connection header received
0926 11:40:26.317516 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:26.317519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:26.317575 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:26.317648 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:26.319110 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.319631 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.321217 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.321539 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.321648 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.325098 (+ 3450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:26.325103 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:26.325106 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:26.325133 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:26.326556 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:26.327134 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:26.327137 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:26.327138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:26.327216 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:26.327474 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:26.327476 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:26.327477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:26.327639 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:26.327704 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:26.328116 (+ 412us) server_negotiation.cc:299] Negotiation successful
0926 11:40:26.328210 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 11:40:26.329073 8635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.315981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:38135 (local address 127.6.96.3:60325)
0926 11:40:26.316396 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:26.316443 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:26.317301 (+ 858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:26.317837 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:26.317846 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:26.318293 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:26.318958 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:26.318978 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.319740 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.319743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:26.321104 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:26.321114 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.321717 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.321722 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:26.321926 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.323158 (+ 1232us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:26.323179 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:26.324936 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:26.326679 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:26.326684 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:26.326697 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:26.327032 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:26.327285 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:26.327287 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:26.327289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:26.327406 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:26.327705 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:26.327709 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:26.327932 (+ 223us) client_negotiation.cc:769] Sending connection context
0926 11:40:26.328071 (+ 139us) client_negotiation.cc:240] Negotiation successful
0926 11:40:26.328309 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":101,"threads_started":1}
I20260926 11:40:26.330152 8742 heartbeater.cc:347] Connected to a master server at 127.6.96.62:38135
I20260926 11:40:26.330330 8742 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:26.330886 8742 heartbeater.cc:511] Master 127.6.96.62:38135 requested a full tablet report, sending...
I20260926 11:40:26.331905 8258 ts_manager.cc:194] Registered new tserver with Master: fa45b8b0831246c5b2fde85e78124b88 (127.6.96.3:36655)
I20260926 11:40:26.332463 8258 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.3@KRBTEST.COM'} at 127.6.96.3:60325
I20260926 11:40:26.335906 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 8611
I20260926 11:40:26.335999 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:40:26.336246 6528 external_mini_cluster.cc:1428] Setting key e791c030a5c453c6bcc34f0a9ab479ef
I20260926 11:40:26.338137 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 11:40:26.339532 8748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.332897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:38135 (local address 127.6.96.3:59359)
0926 11:40:26.333146 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:26.333157 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:26.333238 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:26.333446 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:26.333448 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:26.333606 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:26.333737 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:26.333743 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.334631 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.334633 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:26.335233 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:26.335240 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.335614 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.335616 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:26.335692 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.336208 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:26.336220 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:26.336579 (+ 359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:26.338181 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:26.338184 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:26.338185 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:26.338383 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:26.338625 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:26.338627 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:26.338628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:26.338692 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:26.339102 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:26.339104 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:26.339183 (+ 79us) client_negotiation.cc:769] Sending connection context
0926 11:40:26.339277 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 11:40:26.339388 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":1,"thread_start_us":91,"threads_started":1}
I20260926 11:40:26.339571 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.332930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:59359 (local address 127.6.96.62:38135)
0926 11:40:26.333024 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:26.333027 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:26.333223 (+ 196us) server_negotiation.cc:407] Connection header received
0926 11:40:26.333310 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:26.333313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:26.333357 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:26.333436 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:26.333837 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.334540 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.335330 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.335532 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.335608 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.336724 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:26.336728 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:26.336730 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:26.336753 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:26.338088 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:26.338463 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:26.338466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:26.338468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:26.338548 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:26.338774 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:26.338777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:26.338778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:26.338994 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:26.339078 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:26.339337 (+ 259us) server_negotiation.cc:299] Negotiation successful
0926 11:40:26.339437 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
Sep 26 11:40:26 dist-test-slave-xxxx krb5kdc[8214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 11:40:26 dist-test-slave-xxxx krb5kdc[8214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:26.361933 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.354287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44900 (local address 127.6.96.62:38135)
0926 11:40:26.354406 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:26.354409 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:26.354421 (+ 12us) server_negotiation.cc:407] Connection header received
0926 11:40:26.354458 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:26.354460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:26.354496 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:26.354589 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:26.355470 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.356105 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.356945 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.357153 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.357229 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.359462 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:26.359465 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:26.359467 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:26.359490 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:26.360615 (+ 1125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:26.360968 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:26.360970 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:26.360971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:26.361032 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:26.361210 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:26.361212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:26.361213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:26.361414 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:26.361528 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:26.361699 (+ 171us) server_negotiation.cc:299] Negotiation successful
0926 11:40:26.361784 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 11:40:26.364548 8258 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 11:40:26.367545 8258 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:40:26.381719 8764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.376504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34848 (local address 127.6.96.1:34477)
0926 11:40:26.376836 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:26.376850 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:26.376873 (+ 23us) server_negotiation.cc:407] Connection header received
0926 11:40:26.377028 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:26.377036 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:26.377265 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:26.377416 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:26.377803 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.378990 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.380345 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.381057 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.381157 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.381323 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:26.381472 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 11:40:26.381564 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":93,"threads_started":1}
I20260926 11:40:26.381759 8760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.375946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:34477 (local address 127.0.0.1:34848)
0926 11:40:26.376787 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:26.376799 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:26.376870 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:26.377576 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:26.377578 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:26.377591 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:26.377682 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:26.377687 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.379131 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.379134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:26.380224 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:26.380234 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.381168 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.381172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:26.381296 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.381336 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 11:40:26.381436 (+ 100us) client_negotiation.cc:240] Negotiation successful
0926 11:40:26.381596 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":790,"mutex_wait_us":47,"thread_start_us":77,"threads_started":1}
I20260926 11:40:26.381927 8763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.376418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45216 (local address 127.6.96.2:35251)
0926 11:40:26.377029 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:26.377040 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:26.377060 (+ 20us) server_negotiation.cc:407] Connection header received
0926 11:40:26.377120 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:26.377124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:26.377264 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:26.377416 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:26.377796 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.378552 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.380100 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.381063 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.381161 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.381331 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:26.381701 (+ 370us) server_negotiation.cc:299] Negotiation successful
0926 11:40:26.381790 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":100,"threads_started":1}
I20260926 11:40:26.382076 8761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.375796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45928 (local address 127.6.96.3:36655)
0926 11:40:26.377075 (+ 1279us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:26.377086 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:26.377107 (+ 21us) server_negotiation.cc:407] Connection header received
0926 11:40:26.377177 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:26.377181 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:26.377338 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:26.377637 (+ 299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:26.377786 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.378990 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.380426 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.381184 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.381678 (+ 494us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.381832 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:26.381876 (+ 44us) server_negotiation.cc:299] Negotiation successful
0926 11:40:26.381951 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1180,"thread_start_us":94,"threads_started":1}
I20260926 11:40:26.382176 8762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.375762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:35251 (local address 127.0.0.1:45216)
0926 11:40:26.376494 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:26.376507 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:26.376595 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:26.377420 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:26.377426 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:26.377449 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:26.377663 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:26.377673 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.378659 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.378662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:26.379975 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:26.379995 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.381380 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.381384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:26.381456 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.381474 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 11:40:26.381529 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 11:40:26.381591 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"mutex_wait_us":26,"spinlock_wait_cycles":4736,"thread_start_us":97,"threads_started":1}
I20260926 11:40:26.382191 8759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.375660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:36655 (local address 127.0.0.1:45928)
0926 11:40:26.376428 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:26.376478 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:26.376596 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:26.377470 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:26.377473 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:26.377495 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:26.377663 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:26.377673 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.379114 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.379118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:26.380317 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:26.380326 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.381306 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.381309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:26.381400 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.381422 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 11:40:26.381468 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 11:40:26.381597 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"mutex_wait_us":232,"spinlock_wait_cycles":768,"thread_start_us":85,"threads_started":1}
I20260926 11:40:26.384711 8536 tablet_service.cc:1467] Processing CreateTablet for tablet 471692c598e6455795455bec6c7e2eb2 (DEFAULT_TABLE table=test-table [id=fc6c9c5a56c3485e9b861f326c439242]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:26.384876 8676 tablet_service.cc:1467] Processing CreateTablet for tablet 471692c598e6455795455bec6c7e2eb2 (DEFAULT_TABLE table=test-table [id=fc6c9c5a56c3485e9b861f326c439242]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:26.384876 8396 tablet_service.cc:1467] Processing CreateTablet for tablet 471692c598e6455795455bec6c7e2eb2 (DEFAULT_TABLE table=test-table [id=fc6c9c5a56c3485e9b861f326c439242]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:26.385723 8536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 471692c598e6455795455bec6c7e2eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:26.385838 8396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 471692c598e6455795455bec6c7e2eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:26.385843 8676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 471692c598e6455795455bec6c7e2eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:26.392536 8765 tablet_bootstrap.cc:492] T 471692c598e6455795455bec6c7e2eb2 P ea9ce406518f47f18242a8d987510090: Bootstrap starting.
I20260926 11:40:26.393903 8767 tablet_bootstrap.cc:492] T 471692c598e6455795455bec6c7e2eb2 P fa45b8b0831246c5b2fde85e78124b88: Bootstrap starting.
I20260926 11:40:26.395143 8765 tablet_bootstrap.cc:654] T 471692c598e6455795455bec6c7e2eb2 P ea9ce406518f47f18242a8d987510090: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:26.395154 8766 tablet_bootstrap.cc:492] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d: Bootstrap starting.
I20260926 11:40:26.395825 8767 tablet_bootstrap.cc:654] T 471692c598e6455795455bec6c7e2eb2 P fa45b8b0831246c5b2fde85e78124b88: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:26.395843 8765 log.cc:824] T 471692c598e6455795455bec6c7e2eb2 P ea9ce406518f47f18242a8d987510090: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:26.396663 8767 log.cc:824] T 471692c598e6455795455bec6c7e2eb2 P fa45b8b0831246c5b2fde85e78124b88: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:26.397763 8766 tablet_bootstrap.cc:654] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:26.398651 8766 log.cc:824] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:26.399940 8767 tablet_bootstrap.cc:492] T 471692c598e6455795455bec6c7e2eb2 P fa45b8b0831246c5b2fde85e78124b88: No bootstrap required, opened a new log
I20260926 11:40:26.400121 8767 ts_tablet_manager.cc:1397] T 471692c598e6455795455bec6c7e2eb2 P fa45b8b0831246c5b2fde85e78124b88: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 11:40:26.400409 8765 tablet_bootstrap.cc:492] T 471692c598e6455795455bec6c7e2eb2 P ea9ce406518f47f18242a8d987510090: No bootstrap required, opened a new log
I20260926 11:40:26.400612 8765 ts_tablet_manager.cc:1397] T 471692c598e6455795455bec6c7e2eb2 P ea9ce406518f47f18242a8d987510090: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260926 11:40:26.404614 8766 tablet_bootstrap.cc:492] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d: No bootstrap required, opened a new log
I20260926 11:40:26.404417 8765 raft_consensus.cc:348] T 471692c598e6455795455bec6c7e2eb2 P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } }
I20260926 11:40:26.404768 8765 raft_consensus.cc:374] T 471692c598e6455795455bec6c7e2eb2 P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:26.404837 8766 ts_tablet_manager.cc:1397] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260926 11:40:26.404841 8765 raft_consensus.cc:729] T 471692c598e6455795455bec6c7e2eb2 P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea9ce406518f47f18242a8d987510090, State: Initialized, Role: FOLLOWER
I20260926 11:40:26.404594 8767 raft_consensus.cc:348] T 471692c598e6455795455bec6c7e2eb2 P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } }
I20260926 11:40:26.404996 8767 raft_consensus.cc:374] T 471692c598e6455795455bec6c7e2eb2 P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:26.405069 8767 raft_consensus.cc:729] T 471692c598e6455795455bec6c7e2eb2 P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa45b8b0831246c5b2fde85e78124b88, State: Initialized, Role: FOLLOWER
I20260926 11:40:26.405331 8765 consensus_queue.cc:260] T 471692c598e6455795455bec6c7e2eb2 P ea9ce406518f47f18242a8d987510090 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } }
I20260926 11:40:26.405795 8767 consensus_queue.cc:260] T 471692c598e6455795455bec6c7e2eb2 P fa45b8b0831246c5b2fde85e78124b88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } }
I20260926 11:40:26.406384 8765 ts_tablet_manager.cc:1428] T 471692c598e6455795455bec6c7e2eb2 P ea9ce406518f47f18242a8d987510090: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 11:40:26.406759 8602 heartbeater.cc:503] Master 127.6.96.62:38135 was elected leader, sending a full tablet report...
I20260926 11:40:26.408676 8766 raft_consensus.cc:348] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } }
I20260926 11:40:26.408867 8766 raft_consensus.cc:374] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:26.408941 8766 raft_consensus.cc:729] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a06f0913f7745efa75ccfde9eaf9c8d, State: Initialized, Role: FOLLOWER
I20260926 11:40:26.409426 8766 consensus_queue.cc:260] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } }
I20260926 11:40:26.410348 8766 ts_tablet_manager.cc:1428] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260926 11:40:26.410738 8462 heartbeater.cc:503] Master 127.6.96.62:38135 was elected leader, sending a full tablet report...
I20260926 11:40:26.412365 8767 ts_tablet_manager.cc:1428] T 471692c598e6455795455bec6c7e2eb2 P fa45b8b0831246c5b2fde85e78124b88: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260926 11:40:26.412727 8742 heartbeater.cc:503] Master 127.6.96.62:38135 was elected leader, sending a full tablet report...
W20260926 11:40:26.415364 8463 tablet.cc:2369] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:40:26.550499 8773 raft_consensus.cc:482] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:40:26.550721 8773 raft_consensus.cc:504] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } }
I20260926 11:40:26.551908 8773 leader_election.cc:288] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa45b8b0831246c5b2fde85e78124b88 (127.6.96.3:36655), ea9ce406518f47f18242a8d987510090 (127.6.96.2:35251)
I20260926 11:40:26.556316 8763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.552354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:50175 (local address 127.6.96.2:35251)
0926 11:40:26.552472 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:26.552476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:26.552653 (+ 177us) server_negotiation.cc:407] Connection header received
0926 11:40:26.552763 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:26.552767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:26.552824 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:26.552915 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:26.553318 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.554147 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.555342 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.555730 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.555828 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.555893 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:26.556108 (+ 215us) server_negotiation.cc:299] Negotiation successful
0926 11:40:26.556181 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 11:40:26.556321 8761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.552262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:55525 (local address 127.6.96.3:36655)
0926 11:40:26.552380 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:26.552384 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:26.552729 (+ 345us) server_negotiation.cc:407] Connection header received
0926 11:40:26.552793 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:26.552795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:26.552848 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:26.552921 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:26.553313 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.554150 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.555335 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.555741 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.555836 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.555897 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:26.556115 (+ 218us) server_negotiation.cc:299] Negotiation successful
0926 11:40:26.556194 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 11:40:26.556339 8775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.552177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:35251 (local address 127.6.96.1:50175)
0926 11:40:26.552530 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:26.552546 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:26.552665 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:26.552958 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:26.552962 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:26.553004 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:26.553178 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:26.553186 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.554305 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.554308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:26.555216 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:26.555226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.555846 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.555850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:26.555977 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.556004 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 11:40:26.556077 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 11:40:26.556163 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":127,"spinlock_wait_cycles":25216,"thread_start_us":67,"threads_started":1}
I20260926 11:40:26.556348 8774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.552177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:36655 (local address 127.6.96.1:55525)
0926 11:40:26.552632 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:26.552643 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:26.552727 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:26.552958 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:26.552962 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:26.553006 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:26.553188 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:26.553193 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.554285 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.554289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:26.555214 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:26.555224 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.555851 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.555854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:26.555985 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.556017 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 11:40:26.556089 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 11:40:26.556185 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":115,"spinlock_wait_cycles":7040,"thread_start_us":91,"threads_started":1}
I20260926 11:40:26.557044 8696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "471692c598e6455795455bec6c7e2eb2" candidate_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa45b8b0831246c5b2fde85e78124b88" is_pre_election: true
I20260926 11:40:26.557071 8556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "471692c598e6455795455bec6c7e2eb2" candidate_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea9ce406518f47f18242a8d987510090" is_pre_election: true
I20260926 11:40:26.557329 8556 raft_consensus.cc:2454] T 471692c598e6455795455bec6c7e2eb2 P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a06f0913f7745efa75ccfde9eaf9c8d in term 0.
I20260926 11:40:26.557327 8696 raft_consensus.cc:2454] T 471692c598e6455795455bec6c7e2eb2 P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a06f0913f7745efa75ccfde9eaf9c8d in term 0.
I20260926 11:40:26.557822 8347 leader_election.cc:302] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0a06f0913f7745efa75ccfde9eaf9c8d, ea9ce406518f47f18242a8d987510090, fa45b8b0831246c5b2fde85e78124b88; no voters:
I20260926 11:40:26.558077 8773 raft_consensus.cc:2781] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:40:26.558146 8773 raft_consensus.cc:482] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:40:26.558175 8773 raft_consensus.cc:3037] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:26.559203 8773 raft_consensus.cc:504] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } }
I20260926 11:40:26.559512 8773 leader_election.cc:288] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d [CANDIDATE]: Term 1 election: Requested vote from peers fa45b8b0831246c5b2fde85e78124b88 (127.6.96.3:36655), ea9ce406518f47f18242a8d987510090 (127.6.96.2:35251)
I20260926 11:40:26.559806 8696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "471692c598e6455795455bec6c7e2eb2" candidate_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa45b8b0831246c5b2fde85e78124b88"
I20260926 11:40:26.559847 8556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "471692c598e6455795455bec6c7e2eb2" candidate_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea9ce406518f47f18242a8d987510090"
I20260926 11:40:26.559912 8696 raft_consensus.cc:3037] T 471692c598e6455795455bec6c7e2eb2 P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:26.559973 8556 raft_consensus.cc:3037] T 471692c598e6455795455bec6c7e2eb2 P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:26.560977 8696 raft_consensus.cc:2454] T 471692c598e6455795455bec6c7e2eb2 P fa45b8b0831246c5b2fde85e78124b88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a06f0913f7745efa75ccfde9eaf9c8d in term 1.
I20260926 11:40:26.561081 8556 raft_consensus.cc:2454] T 471692c598e6455795455bec6c7e2eb2 P ea9ce406518f47f18242a8d987510090 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a06f0913f7745efa75ccfde9eaf9c8d in term 1.
I20260926 11:40:26.561242 8347 leader_election.cc:302] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a06f0913f7745efa75ccfde9eaf9c8d, fa45b8b0831246c5b2fde85e78124b88; no voters:
I20260926 11:40:26.561393 8773 raft_consensus.cc:2781] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:26.561568 8773 raft_consensus.cc:686] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 1 LEADER]: Becoming Leader. State: Replica: 0a06f0913f7745efa75ccfde9eaf9c8d, State: Running, Role: LEADER
I20260926 11:40:26.561906 8773 consensus_queue.cc:237] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } }
I20260926 11:40:26.564975 8259 catalog_manager.cc:5495] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a06f0913f7745efa75ccfde9eaf9c8d (127.6.96.1). New cstate: current_term: 1 leader_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } health_report { overall_health: HEALTHY } } }
W20260926 11:40:26.577412 8743 tablet.cc:2369] T 471692c598e6455795455bec6c7e2eb2 P fa45b8b0831246c5b2fde85e78124b88: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 11:40:26.620908 8603 tablet.cc:2369] T 471692c598e6455795455bec6c7e2eb2 P ea9ce406518f47f18242a8d987510090: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 11:40:26 dist-test-slave-xxxx krb5kdc[8214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422825, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:26.643388 8783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.628332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:38135 (local address 127.0.0.1:44914)
0926 11:40:26.628574 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:26.628586 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:26.628697 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:26.628929 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:26.628932 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:26.629359 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:26.629460 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:26.629467 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.630235 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.630238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:26.630886 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:26.630896 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.631297 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.631300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:26.631421 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.638743 (+ 7322us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:26.638757 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:26.640202 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:26.641828 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:26.641840 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:26.641847 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:26.642075 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:26.642323 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:26.642326 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:26.642328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:26.642399 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:26.642764 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:26.642771 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:26.642925 (+ 154us) client_negotiation.cc:769] Sending connection context
0926 11:40:26.643029 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 11:40:26.643189 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":31,"thread_start_us":94,"threads_started":1}
I20260926 11:40:26.643390 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.628393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44914 (local address 127.6.96.62:38135)
0926 11:40:26.628518 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:26.628523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:26.628654 (+ 131us) server_negotiation.cc:407] Connection header received
0926 11:40:26.628799 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:26.628802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:26.628841 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:26.628903 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:26.629619 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.630137 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.630986 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.631190 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.631286 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.640357 (+ 9071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:26.640360 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:26.640362 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:26.640385 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:26.641720 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:26.642167 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:26.642170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:26.642171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:26.642243 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:26.642473 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:26.642475 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:26.642477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:26.642652 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:26.642740 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:26.643085 (+ 345us) server_negotiation.cc:299] Negotiation successful
0926 11:40:26.643284 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":46}
I20260926 11:40:26.647323 8259 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:44914:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 11:40:26.647994 8259 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:40:26.652940 8536 tablet_service.cc:1467] Processing CreateTablet for tablet cb1f1375880b484babd0457b065f883c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8a009d93b4f84c25a013e8bb5682e411]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 11:40:26.652731 8396 tablet_service.cc:1467] Processing CreateTablet for tablet cb1f1375880b484babd0457b065f883c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8a009d93b4f84c25a013e8bb5682e411]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 11:40:26.652972 8676 tablet_service.cc:1467] Processing CreateTablet for tablet cb1f1375880b484babd0457b065f883c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8a009d93b4f84c25a013e8bb5682e411]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 11:40:26.653544 8536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb1f1375880b484babd0457b065f883c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:26.653553 8396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb1f1375880b484babd0457b065f883c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:26.653664 8676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb1f1375880b484babd0457b065f883c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:26.656283 8767 tablet_bootstrap.cc:492] T cb1f1375880b484babd0457b065f883c P fa45b8b0831246c5b2fde85e78124b88: Bootstrap starting.
I20260926 11:40:26.656507 8765 tablet_bootstrap.cc:492] T cb1f1375880b484babd0457b065f883c P ea9ce406518f47f18242a8d987510090: Bootstrap starting.
I20260926 11:40:26.656507 8766 tablet_bootstrap.cc:492] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d: Bootstrap starting.
I20260926 11:40:26.657105 8767 tablet_bootstrap.cc:654] T cb1f1375880b484babd0457b065f883c P fa45b8b0831246c5b2fde85e78124b88: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:26.657436 8765 tablet_bootstrap.cc:654] T cb1f1375880b484babd0457b065f883c P ea9ce406518f47f18242a8d987510090: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:26.657750 8766 tablet_bootstrap.cc:654] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:26.658744 8767 tablet_bootstrap.cc:492] T cb1f1375880b484babd0457b065f883c P fa45b8b0831246c5b2fde85e78124b88: No bootstrap required, opened a new log
I20260926 11:40:26.658794 8765 tablet_bootstrap.cc:492] T cb1f1375880b484babd0457b065f883c P ea9ce406518f47f18242a8d987510090: No bootstrap required, opened a new log
I20260926 11:40:26.658869 8767 ts_tablet_manager.cc:1397] T cb1f1375880b484babd0457b065f883c P fa45b8b0831246c5b2fde85e78124b88: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 11:40:26.658882 8765 ts_tablet_manager.cc:1397] T cb1f1375880b484babd0457b065f883c P ea9ce406518f47f18242a8d987510090: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 11:40:26.659020 8766 tablet_bootstrap.cc:492] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d: No bootstrap required, opened a new log
I20260926 11:40:26.659101 8766 ts_tablet_manager.cc:1397] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 11:40:26.659303 8765 raft_consensus.cc:348] T cb1f1375880b484babd0457b065f883c P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:26.659303 8767 raft_consensus.cc:348] T cb1f1375880b484babd0457b065f883c P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:26.659418 8765 raft_consensus.cc:374] T cb1f1375880b484babd0457b065f883c P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:26.659418 8767 raft_consensus.cc:374] T cb1f1375880b484babd0457b065f883c P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:26.659449 8765 raft_consensus.cc:729] T cb1f1375880b484babd0457b065f883c P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea9ce406518f47f18242a8d987510090, State: Initialized, Role: FOLLOWER
I20260926 11:40:26.659449 8767 raft_consensus.cc:729] T cb1f1375880b484babd0457b065f883c P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa45b8b0831246c5b2fde85e78124b88, State: Initialized, Role: FOLLOWER
I20260926 11:40:26.659482 8766 raft_consensus.cc:348] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:26.659592 8766 raft_consensus.cc:374] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:26.659564 8767 consensus_queue.cc:260] T cb1f1375880b484babd0457b065f883c P fa45b8b0831246c5b2fde85e78124b88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:26.659564 8765 consensus_queue.cc:260] T cb1f1375880b484babd0457b065f883c P ea9ce406518f47f18242a8d987510090 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:26.659628 8766 raft_consensus.cc:729] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a06f0913f7745efa75ccfde9eaf9c8d, State: Initialized, Role: FOLLOWER
I20260926 11:40:26.659751 8766 consensus_queue.cc:260] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:26.659808 8765 ts_tablet_manager.cc:1428] T cb1f1375880b484babd0457b065f883c P ea9ce406518f47f18242a8d987510090: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260926 11:40:26.659811 8767 ts_tablet_manager.cc:1428] T cb1f1375880b484babd0457b065f883c P fa45b8b0831246c5b2fde85e78124b88: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:26.660037 8772 tablet_replica.cc:439] T cb1f1375880b484babd0457b065f883c P fa45b8b0831246c5b2fde85e78124b88: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } } }
I20260926 11:40:26.660169 8772 tablet_replica.cc:442] T cb1f1375880b484babd0457b065f883c P fa45b8b0831246c5b2fde85e78124b88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 11:40:26.660238 8771 tablet_replica.cc:439] T cb1f1375880b484babd0457b065f883c P ea9ce406518f47f18242a8d987510090: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } } }
I20260926 11:40:26.660372 8771 tablet_replica.cc:442] T cb1f1375880b484babd0457b065f883c P ea9ce406518f47f18242a8d987510090: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 11:40:26.660930 8766 ts_tablet_manager.cc:1428] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260926 11:40:26.660923 8773 tablet_replica.cc:439] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } } }
I20260926 11:40:26.661064 8773 tablet_replica.cc:442] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 11:40:26.846138 8773 raft_consensus.cc:482] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:40:26.846276 8773 raft_consensus.cc:504] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:26.846673 8773 leader_election.cc:288] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea9ce406518f47f18242a8d987510090 (127.6.96.2:35251), fa45b8b0831246c5b2fde85e78124b88 (127.6.96.3:36655)
I20260926 11:40:26.847173 8556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb1f1375880b484babd0457b065f883c" candidate_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea9ce406518f47f18242a8d987510090" is_pre_election: true
I20260926 11:40:26.847159 8696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb1f1375880b484babd0457b065f883c" candidate_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa45b8b0831246c5b2fde85e78124b88" is_pre_election: true
I20260926 11:40:26.847319 8696 raft_consensus.cc:2454] T cb1f1375880b484babd0457b065f883c P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a06f0913f7745efa75ccfde9eaf9c8d in term 0.
I20260926 11:40:26.847337 8556 raft_consensus.cc:2454] T cb1f1375880b484babd0457b065f883c P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a06f0913f7745efa75ccfde9eaf9c8d in term 0.
I20260926 11:40:26.847632 8347 leader_election.cc:302] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a06f0913f7745efa75ccfde9eaf9c8d, fa45b8b0831246c5b2fde85e78124b88; no voters:
I20260926 11:40:26.847815 8773 raft_consensus.cc:2781] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:40:26.847875 8773 raft_consensus.cc:482] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:40:26.847915 8773 raft_consensus.cc:3037] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:26.848835 8773 raft_consensus.cc:504] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:26.849118 8773 leader_election.cc:288] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d [CANDIDATE]: Term 1 election: Requested vote from peers ea9ce406518f47f18242a8d987510090 (127.6.96.2:35251), fa45b8b0831246c5b2fde85e78124b88 (127.6.96.3:36655)
I20260926 11:40:26.849471 8556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb1f1375880b484babd0457b065f883c" candidate_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea9ce406518f47f18242a8d987510090"
I20260926 11:40:26.849470 8696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb1f1375880b484babd0457b065f883c" candidate_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa45b8b0831246c5b2fde85e78124b88"
I20260926 11:40:26.849584 8556 raft_consensus.cc:3037] T cb1f1375880b484babd0457b065f883c P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:26.849584 8696 raft_consensus.cc:3037] T cb1f1375880b484babd0457b065f883c P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:26.850630 8696 raft_consensus.cc:2454] T cb1f1375880b484babd0457b065f883c P fa45b8b0831246c5b2fde85e78124b88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a06f0913f7745efa75ccfde9eaf9c8d in term 1.
I20260926 11:40:26.850638 8556 raft_consensus.cc:2454] T cb1f1375880b484babd0457b065f883c P ea9ce406518f47f18242a8d987510090 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a06f0913f7745efa75ccfde9eaf9c8d in term 1.
I20260926 11:40:26.850956 8347 leader_election.cc:302] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0a06f0913f7745efa75ccfde9eaf9c8d, ea9ce406518f47f18242a8d987510090, fa45b8b0831246c5b2fde85e78124b88; no voters:
I20260926 11:40:26.851123 8773 raft_consensus.cc:2781] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:26.851212 8773 raft_consensus.cc:686] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d [term 1 LEADER]: Becoming Leader. State: Replica: 0a06f0913f7745efa75ccfde9eaf9c8d, State: Running, Role: LEADER
I20260926 11:40:26.851372 8773 consensus_queue.cc:237] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:26.852008 8776 tablet_replica.cc:439] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d: TxnStatusTablet state changed. Reason: New leader 0a06f0913f7745efa75ccfde9eaf9c8d. Latest consensus state: current_term: 1 leader_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } } }
I20260926 11:40:26.852113 8776 tablet_replica.cc:442] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d: This TxnStatusTablet replica's current role is: LEADER
I20260926 11:40:26.852411 8788 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 11:40:26.852630 8788 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 11:40:26.853040 8259 catalog_manager.cc:5495] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a06f0913f7745efa75ccfde9eaf9c8d (127.6.96.1). New cstate: current_term: 1 leader_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260926 11:40:26.894218 8783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.890537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:34477 (local address 127.0.0.1:34864)
0926 11:40:26.890694 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:26.890709 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:26.890814 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:26.891189 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:26.891191 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:26.891199 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 11:40:26.891302 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:26.891307 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.892078 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.892082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:26.892703 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:26.892712 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.893153 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.893156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:26.893245 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.893287 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 11:40:26.893933 (+ 646us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 11:40:26.893946 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 11:40:26.894007 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 11:40:26.894094 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 11:40:26.894235 8791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.890636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34864 (local address 127.6.96.1:34477)
0926 11:40:26.890901 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:26.890906 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:26.890921 (+ 15us) server_negotiation.cc:407] Connection header received
0926 11:40:26.891015 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:26.891017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:26.891070 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:26.891161 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:40:26.891398 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.891962 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.892810 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.893057 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.893141 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.893422 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:40:26.893847 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:40:26.893947 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:26.894019 (+ 72us) server_negotiation.cc:299] Negotiation successful
0926 11:40:26.894072 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":77,"threads_started":1}
I20260926 11:40:26.899261 8556 raft_consensus.cc:1261] T cb1f1375880b484babd0457b065f883c P ea9ce406518f47f18242a8d987510090 [term 1 FOLLOWER]: Refusing update from remote peer 0a06f0913f7745efa75ccfde9eaf9c8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:26.899250 8696 raft_consensus.cc:1261] T cb1f1375880b484babd0457b065f883c P fa45b8b0831246c5b2fde85e78124b88 [term 1 FOLLOWER]: Refusing update from remote peer 0a06f0913f7745efa75ccfde9eaf9c8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:26.899950 8773 consensus_queue.cc:1035] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:26.900128 8776 consensus_queue.cc:1035] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:26.903689 8771 tablet_replica.cc:439] T cb1f1375880b484babd0457b065f883c P ea9ce406518f47f18242a8d987510090: TxnStatusTablet state changed. Reason: New leader 0a06f0913f7745efa75ccfde9eaf9c8d. Latest consensus state: current_term: 1 leader_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } } }
I20260926 11:40:26.903805 8771 tablet_replica.cc:442] T cb1f1375880b484babd0457b065f883c P ea9ce406518f47f18242a8d987510090: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 11:40:26.904264 8772 tablet_replica.cc:439] T cb1f1375880b484babd0457b065f883c P fa45b8b0831246c5b2fde85e78124b88: TxnStatusTablet state changed. Reason: New leader 0a06f0913f7745efa75ccfde9eaf9c8d. Latest consensus state: current_term: 1 leader_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } } }
I20260926 11:40:26.904373 8772 tablet_replica.cc:442] T cb1f1375880b484babd0457b065f883c P fa45b8b0831246c5b2fde85e78124b88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 11:40:26.907099 8773 tablet_replica.cc:439] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } } }
I20260926 11:40:26.907204 8773 tablet_replica.cc:442] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d: This TxnStatusTablet replica's current role is: LEADER
I20260926 11:40:26.907306 8792 mvcc.cc:205] Tried to move back new op lower bound from 7333571898972950528 to 7333571898784227328. Current Snapshot: MvccSnapshot[applied={T|T < 7333571898972950528}]
I20260926 11:40:26.907732 8771 tablet_replica.cc:439] T cb1f1375880b484babd0457b065f883c P ea9ce406518f47f18242a8d987510090: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } } }
I20260926 11:40:26.907840 8771 tablet_replica.cc:442] T cb1f1375880b484babd0457b065f883c P ea9ce406518f47f18242a8d987510090: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 11:40:26.907775 8793 mvcc.cc:205] Tried to move back new op lower bound from 7333571898972950528 to 7333571898784227328. Current Snapshot: MvccSnapshot[applied={T|T < 7333571898972950528}]
I20260926 11:40:26.908452 8776 tablet_replica.cc:439] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } } }
I20260926 11:40:26.908568 8776 tablet_replica.cc:442] T cb1f1375880b484babd0457b065f883c P 0a06f0913f7745efa75ccfde9eaf9c8d: This TxnStatusTablet replica's current role is: LEADER
I20260926 11:40:26.909879 8772 tablet_replica.cc:439] T cb1f1375880b484babd0457b065f883c P fa45b8b0831246c5b2fde85e78124b88: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } } }
I20260926 11:40:26.909978 8772 tablet_replica.cc:442] T cb1f1375880b484babd0457b065f883c P fa45b8b0831246c5b2fde85e78124b88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 11:40:26.910780 8794 mvcc.cc:205] Tried to move back new op lower bound from 7333571898972950528 to 7333571898784227328. Current Snapshot: MvccSnapshot[applied={T|T < 7333571898972950528}]
I20260926 11:40:26.917424 8791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.914151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34866 (local address 127.6.96.1:34477)
0926 11:40:26.914260 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:26.914265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:26.914501 (+ 236us) server_negotiation.cc:407] Connection header received
0926 11:40:26.914603 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:26.914607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:26.914651 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:26.914711 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:40:26.914975 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.915477 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.916365 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.916636 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.916692 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.917002 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:40:26.917070 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:40:26.917123 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:26.917225 (+ 102us) server_negotiation.cc:299] Negotiation successful
0926 11:40:26.917274 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 11:40:26.932909 8791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.923957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:58025 (local address 127.6.96.1:34477)
0926 11:40:26.924166 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:26.924169 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:26.929625 (+ 5456us) server_negotiation.cc:407] Connection header received
0926 11:40:26.929743 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:26.929745 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:26.929789 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:26.929858 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:40:26.930079 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.930611 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.931935 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.932135 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.932212 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.932516 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:40:26.932580 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:40:26.932628 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:26.932715 (+ 87us) server_negotiation.cc:299] Negotiation successful
0926 11:40:26.932769 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":159}
I20260926 11:40:26.932960 8804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:26.923905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:34477 (local address 127.6.96.1:58025)
0926 11:40:26.929514 (+ 5609us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:26.929530 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:26.929651 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:26.929885 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:26.929888 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:26.929895 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 11:40:26.929998 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:26.930004 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.930700 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.930704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:26.931829 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:26.931840 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:26.932231 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:26.932235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:26.932319 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:26.932373 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 11:40:26.932645 (+ 272us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 11:40:26.932650 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 11:40:26.932708 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 11:40:26.932792 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5507,"thread_start_us":104,"threads_started":1}
I20260926 11:40:26.939309 8556 raft_consensus.cc:1261] T 471692c598e6455795455bec6c7e2eb2 P ea9ce406518f47f18242a8d987510090 [term 1 FOLLOWER]: Refusing update from remote peer 0a06f0913f7745efa75ccfde9eaf9c8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:26.939421 8696 raft_consensus.cc:1261] T 471692c598e6455795455bec6c7e2eb2 P fa45b8b0831246c5b2fde85e78124b88 [term 1 FOLLOWER]: Refusing update from remote peer 0a06f0913f7745efa75ccfde9eaf9c8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:26.939733 8776 consensus_queue.cc:1035] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:26.939886 8773 consensus_queue.cc:1035] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:26.984800 8258 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:26.984970 8258 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:26.987337 8258 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8: Sending DeleteTablet for 3 replicas of tablet 471692c598e6455795455bec6c7e2eb2
I20260926 11:40:26.988077 8676 tablet_service.cc:1514] Processing DeleteTablet for tablet 471692c598e6455795455bec6c7e2eb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:26 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:45928
I20260926 11:40:26.988142 8536 tablet_service.cc:1514] Processing DeleteTablet for tablet 471692c598e6455795455bec6c7e2eb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:26 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:45216
I20260926 11:40:26.988627 8810 tablet_replica.cc:330] T 471692c598e6455795455bec6c7e2eb2 P ea9ce406518f47f18242a8d987510090: stopping tablet replica
I20260926 11:40:26.988873 8396 tablet_service.cc:1514] Processing DeleteTablet for tablet 471692c598e6455795455bec6c7e2eb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:26 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:34848
I20260926 11:40:26.989250 8810 raft_consensus.cc:2229] T 471692c598e6455795455bec6c7e2eb2 P ea9ce406518f47f18242a8d987510090 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:26.989480 8812 tablet_replica.cc:330] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d: stopping tablet replica
I20260926 11:40:26.989523 8810 raft_consensus.cc:2258] T 471692c598e6455795455bec6c7e2eb2 P ea9ce406518f47f18242a8d987510090 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:26.989573 8258 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 11:40:26.989763 8812 raft_consensus.cc:2229] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 1 LEADER]: Raft consensus shutting down.
W20260926 11:40:26.989995 8258 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:40:26.990119 8812 raft_consensus.cc:2258] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:26.991068 8811 tablet_replica.cc:330] T 471692c598e6455795455bec6c7e2eb2 P fa45b8b0831246c5b2fde85e78124b88: stopping tablet replica
I20260926 11:40:26.991377 8812 ts_tablet_manager.cc:1905] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:26.991428 8811 raft_consensus.cc:2229] T 471692c598e6455795455bec6c7e2eb2 P fa45b8b0831246c5b2fde85e78124b88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:26.991804 8811 raft_consensus.cc:2258] T 471692c598e6455795455bec6c7e2eb2 P fa45b8b0831246c5b2fde85e78124b88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:26.993160 8811 ts_tablet_manager.cc:1905] T 471692c598e6455795455bec6c7e2eb2 P fa45b8b0831246c5b2fde85e78124b88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:26.994499 8812 ts_tablet_manager.cc:1918] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 11:40:26.994596 8812 log.cc:1196] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-0/wal/wals/471692c598e6455795455bec6c7e2eb2
I20260926 11:40:26.994922 8812 ts_tablet_manager.cc:1939] T 471692c598e6455795455bec6c7e2eb2 P 0a06f0913f7745efa75ccfde9eaf9c8d: Deleting consensus metadata
I20260926 11:40:26.995052 8536 tablet_service.cc:1467] Processing CreateTablet for tablet 135956c58b2a4cd28bcdeb372eac67a1 (DEFAULT_TABLE table=test-table [id=36ad358e29e34a3ba3f44f6b70109d24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:26.995316 8536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 135956c58b2a4cd28bcdeb372eac67a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:26.996272 8811 ts_tablet_manager.cc:1918] T 471692c598e6455795455bec6c7e2eb2 P fa45b8b0831246c5b2fde85e78124b88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 11:40:26.996410 8811 log.cc:1196] T 471692c598e6455795455bec6c7e2eb2 P fa45b8b0831246c5b2fde85e78124b88: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/471692c598e6455795455bec6c7e2eb2
I20260926 11:40:26.996709 8811 ts_tablet_manager.cc:1939] T 471692c598e6455795455bec6c7e2eb2 P fa45b8b0831246c5b2fde85e78124b88: Deleting consensus metadata
I20260926 11:40:26.997268 8765 tablet_bootstrap.cc:492] T 135956c58b2a4cd28bcdeb372eac67a1 P ea9ce406518f47f18242a8d987510090: Bootstrap starting.
I20260926 11:40:26.998068 8676 tablet_service.cc:1467] Processing CreateTablet for tablet 135956c58b2a4cd28bcdeb372eac67a1 (DEFAULT_TABLE table=test-table [id=36ad358e29e34a3ba3f44f6b70109d24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:26.998184 8765 tablet_bootstrap.cc:654] T 135956c58b2a4cd28bcdeb372eac67a1 P ea9ce406518f47f18242a8d987510090: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:26.998297 8676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 135956c58b2a4cd28bcdeb372eac67a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:26.998951 8243 catalog_manager.cc:4841] TS fa45b8b0831246c5b2fde85e78124b88 (127.6.96.3:36655): tablet 471692c598e6455795455bec6c7e2eb2 (table test-table [id=fc6c9c5a56c3485e9b861f326c439242]) successfully deleted
I20260926 11:40:26.999327 8244 catalog_manager.cc:4841] TS 0a06f0913f7745efa75ccfde9eaf9c8d (127.6.96.1:34477): tablet 471692c598e6455795455bec6c7e2eb2 (table test-table [id=fc6c9c5a56c3485e9b861f326c439242]) successfully deleted
I20260926 11:40:26.999451 8765 tablet_bootstrap.cc:492] T 135956c58b2a4cd28bcdeb372eac67a1 P ea9ce406518f47f18242a8d987510090: No bootstrap required, opened a new log
I20260926 11:40:26.999519 8765 ts_tablet_manager.cc:1397] T 135956c58b2a4cd28bcdeb372eac67a1 P ea9ce406518f47f18242a8d987510090: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260926 11:40:26.999922 8765 raft_consensus.cc:348] T 135956c58b2a4cd28bcdeb372eac67a1 P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:27.000034 8765 raft_consensus.cc:374] T 135956c58b2a4cd28bcdeb372eac67a1 P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:27.000061 8765 raft_consensus.cc:729] T 135956c58b2a4cd28bcdeb372eac67a1 P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea9ce406518f47f18242a8d987510090, State: Initialized, Role: FOLLOWER
I20260926 11:40:27.000172 8765 consensus_queue.cc:260] T 135956c58b2a4cd28bcdeb372eac67a1 P ea9ce406518f47f18242a8d987510090 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:27.000322 8767 tablet_bootstrap.cc:492] T 135956c58b2a4cd28bcdeb372eac67a1 P fa45b8b0831246c5b2fde85e78124b88: Bootstrap starting.
I20260926 11:40:27.000396 8765 ts_tablet_manager.cc:1428] T 135956c58b2a4cd28bcdeb372eac67a1 P ea9ce406518f47f18242a8d987510090: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260926 11:40:27.001160 8767 tablet_bootstrap.cc:654] T 135956c58b2a4cd28bcdeb372eac67a1 P fa45b8b0831246c5b2fde85e78124b88: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:27.001385 8810 ts_tablet_manager.cc:1905] T 471692c598e6455795455bec6c7e2eb2 P ea9ce406518f47f18242a8d987510090: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:27.001907 8396 tablet_service.cc:1467] Processing CreateTablet for tablet 135956c58b2a4cd28bcdeb372eac67a1 (DEFAULT_TABLE table=test-table [id=36ad358e29e34a3ba3f44f6b70109d24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:27.002099 8396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 135956c58b2a4cd28bcdeb372eac67a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:27.003255 8767 tablet_bootstrap.cc:492] T 135956c58b2a4cd28bcdeb372eac67a1 P fa45b8b0831246c5b2fde85e78124b88: No bootstrap required, opened a new log
I20260926 11:40:27.003348 8767 ts_tablet_manager.cc:1397] T 135956c58b2a4cd28bcdeb372eac67a1 P fa45b8b0831246c5b2fde85e78124b88: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 11:40:27.003765 8767 raft_consensus.cc:348] T 135956c58b2a4cd28bcdeb372eac67a1 P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:27.003865 8767 raft_consensus.cc:374] T 135956c58b2a4cd28bcdeb372eac67a1 P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:27.003894 8767 raft_consensus.cc:729] T 135956c58b2a4cd28bcdeb372eac67a1 P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa45b8b0831246c5b2fde85e78124b88, State: Initialized, Role: FOLLOWER
I20260926 11:40:27.004006 8767 consensus_queue.cc:260] T 135956c58b2a4cd28bcdeb372eac67a1 P fa45b8b0831246c5b2fde85e78124b88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:27.004072 8766 tablet_bootstrap.cc:492] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d: Bootstrap starting.
I20260926 11:40:27.004258 8767 ts_tablet_manager.cc:1428] T 135956c58b2a4cd28bcdeb372eac67a1 P fa45b8b0831246c5b2fde85e78124b88: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:27.004595 8810 ts_tablet_manager.cc:1918] T 471692c598e6455795455bec6c7e2eb2 P ea9ce406518f47f18242a8d987510090: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 11:40:27.004751 8810 log.cc:1196] T 471692c598e6455795455bec6c7e2eb2 P ea9ce406518f47f18242a8d987510090: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-1/wal/wals/471692c598e6455795455bec6c7e2eb2
I20260926 11:40:27.004926 8766 tablet_bootstrap.cc:654] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:27.005218 8810 ts_tablet_manager.cc:1939] T 471692c598e6455795455bec6c7e2eb2 P ea9ce406518f47f18242a8d987510090: Deleting consensus metadata
I20260926 11:40:27.006372 8246 catalog_manager.cc:4841] TS ea9ce406518f47f18242a8d987510090 (127.6.96.2:35251): tablet 471692c598e6455795455bec6c7e2eb2 (table test-table [id=fc6c9c5a56c3485e9b861f326c439242]) successfully deleted
I20260926 11:40:27.006618 8766 tablet_bootstrap.cc:492] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d: No bootstrap required, opened a new log
I20260926 11:40:27.006702 8766 ts_tablet_manager.cc:1397] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 11:40:27.007109 8766 raft_consensus.cc:348] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:27.007221 8766 raft_consensus.cc:374] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:27.007252 8766 raft_consensus.cc:729] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a06f0913f7745efa75ccfde9eaf9c8d, State: Initialized, Role: FOLLOWER
I20260926 11:40:27.007365 8766 consensus_queue.cc:260] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:27.007632 8766 ts_tablet_manager.cc:1428] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:27.322394 8808 raft_consensus.cc:482] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:40:27.322528 8808 raft_consensus.cc:504] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:27.322975 8808 leader_election.cc:288] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea9ce406518f47f18242a8d987510090 (127.6.96.2:35251), fa45b8b0831246c5b2fde85e78124b88 (127.6.96.3:36655)
I20260926 11:40:27.323433 8696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "135956c58b2a4cd28bcdeb372eac67a1" candidate_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa45b8b0831246c5b2fde85e78124b88" is_pre_election: true
I20260926 11:40:27.323433 8556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "135956c58b2a4cd28bcdeb372eac67a1" candidate_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea9ce406518f47f18242a8d987510090" is_pre_election: true
I20260926 11:40:27.323592 8696 raft_consensus.cc:2454] T 135956c58b2a4cd28bcdeb372eac67a1 P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a06f0913f7745efa75ccfde9eaf9c8d in term 0.
I20260926 11:40:27.323592 8556 raft_consensus.cc:2454] T 135956c58b2a4cd28bcdeb372eac67a1 P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a06f0913f7745efa75ccfde9eaf9c8d in term 0.
I20260926 11:40:27.323882 8350 leader_election.cc:302] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a06f0913f7745efa75ccfde9eaf9c8d, ea9ce406518f47f18242a8d987510090; no voters:
I20260926 11:40:27.324066 8808 raft_consensus.cc:2781] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:40:27.324128 8808 raft_consensus.cc:482] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:40:27.324155 8808 raft_consensus.cc:3037] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:27.325453 8808 raft_consensus.cc:504] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:27.325776 8808 leader_election.cc:288] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d [CANDIDATE]: Term 1 election: Requested vote from peers ea9ce406518f47f18242a8d987510090 (127.6.96.2:35251), fa45b8b0831246c5b2fde85e78124b88 (127.6.96.3:36655)
I20260926 11:40:27.326097 8556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "135956c58b2a4cd28bcdeb372eac67a1" candidate_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea9ce406518f47f18242a8d987510090"
I20260926 11:40:27.326141 8696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "135956c58b2a4cd28bcdeb372eac67a1" candidate_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa45b8b0831246c5b2fde85e78124b88"
I20260926 11:40:27.326215 8556 raft_consensus.cc:3037] T 135956c58b2a4cd28bcdeb372eac67a1 P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:27.326261 8696 raft_consensus.cc:3037] T 135956c58b2a4cd28bcdeb372eac67a1 P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:27.327168 8556 raft_consensus.cc:2454] T 135956c58b2a4cd28bcdeb372eac67a1 P ea9ce406518f47f18242a8d987510090 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a06f0913f7745efa75ccfde9eaf9c8d in term 1.
I20260926 11:40:27.327318 8696 raft_consensus.cc:2454] T 135956c58b2a4cd28bcdeb372eac67a1 P fa45b8b0831246c5b2fde85e78124b88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a06f0913f7745efa75ccfde9eaf9c8d in term 1.
I20260926 11:40:27.327430 8350 leader_election.cc:302] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a06f0913f7745efa75ccfde9eaf9c8d, ea9ce406518f47f18242a8d987510090; no voters:
I20260926 11:40:27.327598 8808 raft_consensus.cc:2781] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:27.327682 8808 raft_consensus.cc:686] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 1 LEADER]: Becoming Leader. State: Replica: 0a06f0913f7745efa75ccfde9eaf9c8d, State: Running, Role: LEADER
I20260926 11:40:27.327832 8808 consensus_queue.cc:237] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:27.329456 8258 catalog_manager.cc:5495] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a06f0913f7745efa75ccfde9eaf9c8d (127.6.96.1). New cstate: current_term: 1 leader_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260926 11:40:27.385663 8556 raft_consensus.cc:1261] T 135956c58b2a4cd28bcdeb372eac67a1 P ea9ce406518f47f18242a8d987510090 [term 1 FOLLOWER]: Refusing update from remote peer 0a06f0913f7745efa75ccfde9eaf9c8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:27.385646 8696 raft_consensus.cc:1261] T 135956c58b2a4cd28bcdeb372eac67a1 P fa45b8b0831246c5b2fde85e78124b88 [term 1 FOLLOWER]: Refusing update from remote peer 0a06f0913f7745efa75ccfde9eaf9c8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:27.385983 8808 consensus_queue.cc:1035] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:27.386101 8773 consensus_queue.cc:1035] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 11:40:27.398281 8792 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 11:40:28.946138 8259 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:28.946283 8259 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:28.948410 8259 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8: Sending DeleteTablet for 3 replicas of tablet 135956c58b2a4cd28bcdeb372eac67a1
I20260926 11:40:28.948892 8395 tablet_service.cc:1514] Processing DeleteTablet for tablet 135956c58b2a4cd28bcdeb372eac67a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:28 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:34848
I20260926 11:40:28.949005 8536 tablet_service.cc:1514] Processing DeleteTablet for tablet 135956c58b2a4cd28bcdeb372eac67a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:28 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:45216
I20260926 11:40:28.949028 8676 tablet_service.cc:1514] Processing DeleteTablet for tablet 135956c58b2a4cd28bcdeb372eac67a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:28 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:45928
I20260926 11:40:28.949331 8840 tablet_replica.cc:330] T 135956c58b2a4cd28bcdeb372eac67a1 P ea9ce406518f47f18242a8d987510090: stopping tablet replica
I20260926 11:40:28.949331 8839 tablet_replica.cc:330] T 135956c58b2a4cd28bcdeb372eac67a1 P fa45b8b0831246c5b2fde85e78124b88: stopping tablet replica
I20260926 11:40:28.949498 8839 raft_consensus.cc:2229] T 135956c58b2a4cd28bcdeb372eac67a1 P fa45b8b0831246c5b2fde85e78124b88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:28.949659 8839 raft_consensus.cc:2258] T 135956c58b2a4cd28bcdeb372eac67a1 P fa45b8b0831246c5b2fde85e78124b88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:28.950037 8840 raft_consensus.cc:2229] T 135956c58b2a4cd28bcdeb372eac67a1 P ea9ce406518f47f18242a8d987510090 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:28.950210 8840 raft_consensus.cc:2258] T 135956c58b2a4cd28bcdeb372eac67a1 P ea9ce406518f47f18242a8d987510090 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:28.950465 8838 tablet_replica.cc:330] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d: stopping tablet replica
I20260926 11:40:28.950469 8839 ts_tablet_manager.cc:1905] T 135956c58b2a4cd28bcdeb372eac67a1 P fa45b8b0831246c5b2fde85e78124b88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:28.951256 8840 ts_tablet_manager.cc:1905] T 135956c58b2a4cd28bcdeb372eac67a1 P ea9ce406518f47f18242a8d987510090: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:28.951824 8838 raft_consensus.cc:2229] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 1 LEADER]: Raft consensus shutting down.
I20260926 11:40:28.952072 8838 raft_consensus.cc:2258] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:28.952625 8838 ts_tablet_manager.cc:1905] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:28.953009 8839 ts_tablet_manager.cc:1918] T 135956c58b2a4cd28bcdeb372eac67a1 P fa45b8b0831246c5b2fde85e78124b88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 11:40:28.953115 8839 log.cc:1196] T 135956c58b2a4cd28bcdeb372eac67a1 P fa45b8b0831246c5b2fde85e78124b88: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/135956c58b2a4cd28bcdeb372eac67a1
I20260926 11:40:28.953408 8839 ts_tablet_manager.cc:1939] T 135956c58b2a4cd28bcdeb372eac67a1 P fa45b8b0831246c5b2fde85e78124b88: Deleting consensus metadata
I20260926 11:40:28.953888 8840 ts_tablet_manager.cc:1918] T 135956c58b2a4cd28bcdeb372eac67a1 P ea9ce406518f47f18242a8d987510090: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 11:40:28.953987 8840 log.cc:1196] T 135956c58b2a4cd28bcdeb372eac67a1 P ea9ce406518f47f18242a8d987510090: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-1/wal/wals/135956c58b2a4cd28bcdeb372eac67a1
I20260926 11:40:28.954144 8243 catalog_manager.cc:4841] TS fa45b8b0831246c5b2fde85e78124b88 (127.6.96.3:36655): tablet 135956c58b2a4cd28bcdeb372eac67a1 (table test-table [id=36ad358e29e34a3ba3f44f6b70109d24]) successfully deleted
I20260926 11:40:28.954227 8840 ts_tablet_manager.cc:1939] T 135956c58b2a4cd28bcdeb372eac67a1 P ea9ce406518f47f18242a8d987510090: Deleting consensus metadata
I20260926 11:40:28.954777 8246 catalog_manager.cc:4841] TS ea9ce406518f47f18242a8d987510090 (127.6.96.2:35251): tablet 135956c58b2a4cd28bcdeb372eac67a1 (table test-table [id=36ad358e29e34a3ba3f44f6b70109d24]) successfully deleted
I20260926 11:40:28.955212 8838 ts_tablet_manager.cc:1918] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 11:40:28.955284 8838 log.cc:1196] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-0/wal/wals/135956c58b2a4cd28bcdeb372eac67a1
I20260926 11:40:28.955533 8838 ts_tablet_manager.cc:1939] T 135956c58b2a4cd28bcdeb372eac67a1 P 0a06f0913f7745efa75ccfde9eaf9c8d: Deleting consensus metadata
I20260926 11:40:28.956121 8244 catalog_manager.cc:4841] TS 0a06f0913f7745efa75ccfde9eaf9c8d (127.6.96.1:34477): tablet 135956c58b2a4cd28bcdeb372eac67a1 (table test-table [id=36ad358e29e34a3ba3f44f6b70109d24]) successfully deleted
Sep 26 11:40:28 dist-test-slave-xxxx krb5kdc[8214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422828, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 11:40:28 dist-test-slave-xxxx krb5kdc[8214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422828, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:28.974027 8848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:28.966069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44920 (local address 127.6.96.62:38135)
0926 11:40:28.966318 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:28.966324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:28.966340 (+ 16us) server_negotiation.cc:407] Connection header received
0926 11:40:28.966472 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:28.966475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:28.966518 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:28.966600 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:28.967398 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:28.967968 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:28.968836 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:28.969071 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:28.969157 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:28.971397 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:28.971400 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:28.971408 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:28.971432 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:28.972717 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:28.973084 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:28.973086 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:28.973087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:28.973152 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:28.973342 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:28.973344 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:28.973345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:28.973527 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:28.973622 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:28.973807 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 11:40:28.973897 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":92,"threads_started":1}
I20260926 11:40:28.976438 8259 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 11:40:28.976845 8259 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:40:28.981050 8395 tablet_service.cc:1467] Processing CreateTablet for tablet 76e2064747f34a2ab122fdba6a3679af (DEFAULT_TABLE table=test-table [id=408f58e8514047e0ade75b5feb21866a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:28.981242 8676 tablet_service.cc:1467] Processing CreateTablet for tablet 76e2064747f34a2ab122fdba6a3679af (DEFAULT_TABLE table=test-table [id=408f58e8514047e0ade75b5feb21866a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:28.981263 8536 tablet_service.cc:1467] Processing CreateTablet for tablet 76e2064747f34a2ab122fdba6a3679af (DEFAULT_TABLE table=test-table [id=408f58e8514047e0ade75b5feb21866a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:28.981362 8395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76e2064747f34a2ab122fdba6a3679af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:28.981509 8536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76e2064747f34a2ab122fdba6a3679af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:28.981510 8676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76e2064747f34a2ab122fdba6a3679af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:28.983951 8851 tablet_bootstrap.cc:492] T 76e2064747f34a2ab122fdba6a3679af P fa45b8b0831246c5b2fde85e78124b88: Bootstrap starting.
I20260926 11:40:28.983951 8850 tablet_bootstrap.cc:492] T 76e2064747f34a2ab122fdba6a3679af P ea9ce406518f47f18242a8d987510090: Bootstrap starting.
I20260926 11:40:28.984102 8849 tablet_bootstrap.cc:492] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d: Bootstrap starting.
I20260926 11:40:28.984807 8849 tablet_bootstrap.cc:654] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:28.984822 8851 tablet_bootstrap.cc:654] T 76e2064747f34a2ab122fdba6a3679af P fa45b8b0831246c5b2fde85e78124b88: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:28.984833 8850 tablet_bootstrap.cc:654] T 76e2064747f34a2ab122fdba6a3679af P ea9ce406518f47f18242a8d987510090: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:28.985845 8849 tablet_bootstrap.cc:492] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d: No bootstrap required, opened a new log
I20260926 11:40:28.985944 8849 ts_tablet_manager.cc:1397] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 11:40:28.986196 8851 tablet_bootstrap.cc:492] T 76e2064747f34a2ab122fdba6a3679af P fa45b8b0831246c5b2fde85e78124b88: No bootstrap required, opened a new log
I20260926 11:40:28.986280 8851 ts_tablet_manager.cc:1397] T 76e2064747f34a2ab122fdba6a3679af P fa45b8b0831246c5b2fde85e78124b88: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:28.986287 8849 raft_consensus.cc:348] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:28.986398 8849 raft_consensus.cc:374] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:28.986428 8849 raft_consensus.cc:729] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a06f0913f7745efa75ccfde9eaf9c8d, State: Initialized, Role: FOLLOWER
I20260926 11:40:28.986544 8849 consensus_queue.cc:260] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:28.986917 8849 ts_tablet_manager.cc:1428] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:28.986716 8851 raft_consensus.cc:348] T 76e2064747f34a2ab122fdba6a3679af P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:28.987187 8851 raft_consensus.cc:374] T 76e2064747f34a2ab122fdba6a3679af P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:28.987239 8851 raft_consensus.cc:729] T 76e2064747f34a2ab122fdba6a3679af P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa45b8b0831246c5b2fde85e78124b88, State: Initialized, Role: FOLLOWER
I20260926 11:40:28.987398 8851 consensus_queue.cc:260] T 76e2064747f34a2ab122fdba6a3679af P fa45b8b0831246c5b2fde85e78124b88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:28.987735 8850 tablet_bootstrap.cc:492] T 76e2064747f34a2ab122fdba6a3679af P ea9ce406518f47f18242a8d987510090: No bootstrap required, opened a new log
I20260926 11:40:28.987804 8851 ts_tablet_manager.cc:1428] T 76e2064747f34a2ab122fdba6a3679af P fa45b8b0831246c5b2fde85e78124b88: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:28.987877 8850 ts_tablet_manager.cc:1397] T 76e2064747f34a2ab122fdba6a3679af P ea9ce406518f47f18242a8d987510090: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260926 11:40:28.988384 8850 raft_consensus.cc:348] T 76e2064747f34a2ab122fdba6a3679af P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:28.988533 8850 raft_consensus.cc:374] T 76e2064747f34a2ab122fdba6a3679af P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:28.988603 8850 raft_consensus.cc:729] T 76e2064747f34a2ab122fdba6a3679af P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea9ce406518f47f18242a8d987510090, State: Initialized, Role: FOLLOWER
I20260926 11:40:28.988747 8850 consensus_queue.cc:260] T 76e2064747f34a2ab122fdba6a3679af P ea9ce406518f47f18242a8d987510090 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:28.989096 8850 ts_tablet_manager.cc:1428] T 76e2064747f34a2ab122fdba6a3679af P ea9ce406518f47f18242a8d987510090: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:29.128280 8821 raft_consensus.cc:482] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:40:29.128429 8821 raft_consensus.cc:504] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:29.128852 8821 leader_election.cc:288] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea9ce406518f47f18242a8d987510090 (127.6.96.2:35251), fa45b8b0831246c5b2fde85e78124b88 (127.6.96.3:36655)
I20260926 11:40:29.129299 8556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76e2064747f34a2ab122fdba6a3679af" candidate_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea9ce406518f47f18242a8d987510090" is_pre_election: true
I20260926 11:40:29.129298 8696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76e2064747f34a2ab122fdba6a3679af" candidate_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa45b8b0831246c5b2fde85e78124b88" is_pre_election: true
I20260926 11:40:29.129451 8556 raft_consensus.cc:2454] T 76e2064747f34a2ab122fdba6a3679af P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a06f0913f7745efa75ccfde9eaf9c8d in term 0.
I20260926 11:40:29.129470 8696 raft_consensus.cc:2454] T 76e2064747f34a2ab122fdba6a3679af P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a06f0913f7745efa75ccfde9eaf9c8d in term 0.
I20260926 11:40:29.129760 8350 leader_election.cc:302] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a06f0913f7745efa75ccfde9eaf9c8d, ea9ce406518f47f18242a8d987510090; no voters:
I20260926 11:40:29.129997 8821 raft_consensus.cc:2781] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:40:29.130062 8821 raft_consensus.cc:482] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:40:29.130110 8821 raft_consensus.cc:3037] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:29.131433 8821 raft_consensus.cc:504] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:29.131765 8821 leader_election.cc:288] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d [CANDIDATE]: Term 1 election: Requested vote from peers ea9ce406518f47f18242a8d987510090 (127.6.96.2:35251), fa45b8b0831246c5b2fde85e78124b88 (127.6.96.3:36655)
I20260926 11:40:29.132073 8556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76e2064747f34a2ab122fdba6a3679af" candidate_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea9ce406518f47f18242a8d987510090"
I20260926 11:40:29.132091 8696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76e2064747f34a2ab122fdba6a3679af" candidate_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa45b8b0831246c5b2fde85e78124b88"
I20260926 11:40:29.132185 8556 raft_consensus.cc:3037] T 76e2064747f34a2ab122fdba6a3679af P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:29.132193 8696 raft_consensus.cc:3037] T 76e2064747f34a2ab122fdba6a3679af P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:29.133029 8556 raft_consensus.cc:2454] T 76e2064747f34a2ab122fdba6a3679af P ea9ce406518f47f18242a8d987510090 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a06f0913f7745efa75ccfde9eaf9c8d in term 1.
I20260926 11:40:29.133034 8696 raft_consensus.cc:2454] T 76e2064747f34a2ab122fdba6a3679af P fa45b8b0831246c5b2fde85e78124b88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a06f0913f7745efa75ccfde9eaf9c8d in term 1.
I20260926 11:40:29.133294 8347 leader_election.cc:302] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0a06f0913f7745efa75ccfde9eaf9c8d, ea9ce406518f47f18242a8d987510090, fa45b8b0831246c5b2fde85e78124b88; no voters:
I20260926 11:40:29.133469 8821 raft_consensus.cc:2781] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:29.133587 8821 raft_consensus.cc:686] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d [term 1 LEADER]: Becoming Leader. State: Replica: 0a06f0913f7745efa75ccfde9eaf9c8d, State: Running, Role: LEADER
I20260926 11:40:29.133723 8821 consensus_queue.cc:237] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:29.135267 8259 catalog_manager.cc:5495] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a06f0913f7745efa75ccfde9eaf9c8d (127.6.96.1). New cstate: current_term: 1 leader_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260926 11:40:29.179909 8859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:29.176802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34878 (local address 127.6.96.1:34477)
0926 11:40:29.177070 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:29.177079 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:29.177093 (+ 14us) server_negotiation.cc:407] Connection header received
0926 11:40:29.177136 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:29.177139 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:29.177190 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:29.177277 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:40:29.177524 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:29.178110 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:29.179056 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:29.179288 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:29.179357 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:29.179577 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:40:29.179643 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:40:29.179690 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:29.179752 (+ 62us) server_negotiation.cc:299] Negotiation successful
0926 11:40:29.179794 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":104,"threads_started":1}
I20260926 11:40:29.185418 8696 raft_consensus.cc:1261] T 76e2064747f34a2ab122fdba6a3679af P fa45b8b0831246c5b2fde85e78124b88 [term 1 FOLLOWER]: Refusing update from remote peer 0a06f0913f7745efa75ccfde9eaf9c8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:29.185508 8556 raft_consensus.cc:1261] T 76e2064747f34a2ab122fdba6a3679af P ea9ce406518f47f18242a8d987510090 [term 1 FOLLOWER]: Refusing update from remote peer 0a06f0913f7745efa75ccfde9eaf9c8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:29.185752 8821 consensus_queue.cc:1035] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:29.185912 8805 consensus_queue.cc:1035] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:29.215207 8257 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:29.215358 8257 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:29.217234 8257 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8: Sending DeleteTablet for 3 replicas of tablet 76e2064747f34a2ab122fdba6a3679af
I20260926 11:40:29.217795 8396 tablet_service.cc:1514] Processing DeleteTablet for tablet 76e2064747f34a2ab122fdba6a3679af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:29 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:34848
I20260926 11:40:29.217998 8838 tablet_replica.cc:330] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d: stopping tablet replica
I20260926 11:40:29.217970 8676 tablet_service.cc:1514] Processing DeleteTablet for tablet 76e2064747f34a2ab122fdba6a3679af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:29 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:45928
I20260926 11:40:29.218005 8536 tablet_service.cc:1514] Processing DeleteTablet for tablet 76e2064747f34a2ab122fdba6a3679af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:29 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:45216
I20260926 11:40:29.218122 8838 raft_consensus.cc:2229] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d [term 1 LEADER]: Raft consensus shutting down.
I20260926 11:40:29.218155 8839 tablet_replica.cc:330] T 76e2064747f34a2ab122fdba6a3679af P fa45b8b0831246c5b2fde85e78124b88: stopping tablet replica
I20260926 11:40:29.218178 8840 tablet_replica.cc:330] T 76e2064747f34a2ab122fdba6a3679af P ea9ce406518f47f18242a8d987510090: stopping tablet replica
I20260926 11:40:29.218259 8839 raft_consensus.cc:2229] T 76e2064747f34a2ab122fdba6a3679af P fa45b8b0831246c5b2fde85e78124b88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:29.218279 8840 raft_consensus.cc:2229] T 76e2064747f34a2ab122fdba6a3679af P ea9ce406518f47f18242a8d987510090 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:29.218309 8838 raft_consensus.cc:2258] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:29.218353 8839 raft_consensus.cc:2258] T 76e2064747f34a2ab122fdba6a3679af P fa45b8b0831246c5b2fde85e78124b88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:29.218390 8840 raft_consensus.cc:2258] T 76e2064747f34a2ab122fdba6a3679af P ea9ce406518f47f18242a8d987510090 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:29.219023 8840 ts_tablet_manager.cc:1905] T 76e2064747f34a2ab122fdba6a3679af P ea9ce406518f47f18242a8d987510090: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:29.219084 8838 ts_tablet_manager.cc:1905] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:29.219153 8839 ts_tablet_manager.cc:1905] T 76e2064747f34a2ab122fdba6a3679af P fa45b8b0831246c5b2fde85e78124b88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:29.221117 8838 ts_tablet_manager.cc:1918] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 11:40:29.221185 8838 log.cc:1196] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-0/wal/wals/76e2064747f34a2ab122fdba6a3679af
I20260926 11:40:29.221357 8840 ts_tablet_manager.cc:1918] T 76e2064747f34a2ab122fdba6a3679af P ea9ce406518f47f18242a8d987510090: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 11:40:29.221357 8839 ts_tablet_manager.cc:1918] T 76e2064747f34a2ab122fdba6a3679af P fa45b8b0831246c5b2fde85e78124b88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 11:40:29.221393 8838 ts_tablet_manager.cc:1939] T 76e2064747f34a2ab122fdba6a3679af P 0a06f0913f7745efa75ccfde9eaf9c8d: Deleting consensus metadata
I20260926 11:40:29.221433 8840 log.cc:1196] T 76e2064747f34a2ab122fdba6a3679af P ea9ce406518f47f18242a8d987510090: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-1/wal/wals/76e2064747f34a2ab122fdba6a3679af
I20260926 11:40:29.221433 8839 log.cc:1196] T 76e2064747f34a2ab122fdba6a3679af P fa45b8b0831246c5b2fde85e78124b88: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/76e2064747f34a2ab122fdba6a3679af
I20260926 11:40:29.221679 8840 ts_tablet_manager.cc:1939] T 76e2064747f34a2ab122fdba6a3679af P ea9ce406518f47f18242a8d987510090: Deleting consensus metadata
I20260926 11:40:29.221680 8839 ts_tablet_manager.cc:1939] T 76e2064747f34a2ab122fdba6a3679af P fa45b8b0831246c5b2fde85e78124b88: Deleting consensus metadata
I20260926 11:40:29.222151 8244 catalog_manager.cc:4841] TS 0a06f0913f7745efa75ccfde9eaf9c8d (127.6.96.1:34477): tablet 76e2064747f34a2ab122fdba6a3679af (table test-table [id=408f58e8514047e0ade75b5feb21866a]) successfully deleted
I20260926 11:40:29.222272 8257 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 11:40:29.222702 8243 catalog_manager.cc:4841] TS fa45b8b0831246c5b2fde85e78124b88 (127.6.96.3:36655): tablet 76e2064747f34a2ab122fdba6a3679af (table test-table [id=408f58e8514047e0ade75b5feb21866a]) successfully deleted
W20260926 11:40:29.222715 8257 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:40:29.222784 8246 catalog_manager.cc:4841] TS ea9ce406518f47f18242a8d987510090 (127.6.96.2:35251): tablet 76e2064747f34a2ab122fdba6a3679af (table test-table [id=408f58e8514047e0ade75b5feb21866a]) successfully deleted
I20260926 11:40:29.227126 8396 tablet_service.cc:1467] Processing CreateTablet for tablet 4f5c75fa3f3b46519c60190b51d4a568 (DEFAULT_TABLE table=test-table [id=ab8383631a964bf29e895eed611689c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:29.227169 8536 tablet_service.cc:1467] Processing CreateTablet for tablet 4f5c75fa3f3b46519c60190b51d4a568 (DEFAULT_TABLE table=test-table [id=ab8383631a964bf29e895eed611689c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:29.227415 8396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f5c75fa3f3b46519c60190b51d4a568. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:29.227412 8536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f5c75fa3f3b46519c60190b51d4a568. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:29.229256 8850 tablet_bootstrap.cc:492] T 4f5c75fa3f3b46519c60190b51d4a568 P ea9ce406518f47f18242a8d987510090: Bootstrap starting.
I20260926 11:40:29.229331 8849 tablet_bootstrap.cc:492] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d: Bootstrap starting.
I20260926 11:40:29.229656 8676 tablet_service.cc:1467] Processing CreateTablet for tablet 4f5c75fa3f3b46519c60190b51d4a568 (DEFAULT_TABLE table=test-table [id=ab8383631a964bf29e895eed611689c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:29.229900 8676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f5c75fa3f3b46519c60190b51d4a568. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:29.230171 8849 tablet_bootstrap.cc:654] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:29.230176 8850 tablet_bootstrap.cc:654] T 4f5c75fa3f3b46519c60190b51d4a568 P ea9ce406518f47f18242a8d987510090: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:29.231210 8849 tablet_bootstrap.cc:492] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d: No bootstrap required, opened a new log
I20260926 11:40:29.231302 8849 ts_tablet_manager.cc:1397] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 11:40:29.231448 8850 tablet_bootstrap.cc:492] T 4f5c75fa3f3b46519c60190b51d4a568 P ea9ce406518f47f18242a8d987510090: No bootstrap required, opened a new log
I20260926 11:40:29.231534 8850 ts_tablet_manager.cc:1397] T 4f5c75fa3f3b46519c60190b51d4a568 P ea9ce406518f47f18242a8d987510090: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 11:40:29.231690 8849 raft_consensus.cc:348] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:29.231810 8849 raft_consensus.cc:374] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:29.231851 8849 raft_consensus.cc:729] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a06f0913f7745efa75ccfde9eaf9c8d, State: Initialized, Role: FOLLOWER
I20260926 11:40:29.231899 8850 raft_consensus.cc:348] T 4f5c75fa3f3b46519c60190b51d4a568 P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:29.232003 8850 raft_consensus.cc:374] T 4f5c75fa3f3b46519c60190b51d4a568 P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:29.231976 8849 consensus_queue.cc:260] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:29.232039 8850 raft_consensus.cc:729] T 4f5c75fa3f3b46519c60190b51d4a568 P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea9ce406518f47f18242a8d987510090, State: Initialized, Role: FOLLOWER
I20260926 11:40:29.232167 8851 tablet_bootstrap.cc:492] T 4f5c75fa3f3b46519c60190b51d4a568 P fa45b8b0831246c5b2fde85e78124b88: Bootstrap starting.
I20260926 11:40:29.232152 8850 consensus_queue.cc:260] T 4f5c75fa3f3b46519c60190b51d4a568 P ea9ce406518f47f18242a8d987510090 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:29.232223 8849 ts_tablet_manager.cc:1428] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:29.232388 8850 ts_tablet_manager.cc:1428] T 4f5c75fa3f3b46519c60190b51d4a568 P ea9ce406518f47f18242a8d987510090: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:29.233063 8851 tablet_bootstrap.cc:654] T 4f5c75fa3f3b46519c60190b51d4a568 P fa45b8b0831246c5b2fde85e78124b88: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:29.234212 8851 tablet_bootstrap.cc:492] T 4f5c75fa3f3b46519c60190b51d4a568 P fa45b8b0831246c5b2fde85e78124b88: No bootstrap required, opened a new log
I20260926 11:40:29.234282 8851 ts_tablet_manager.cc:1397] T 4f5c75fa3f3b46519c60190b51d4a568 P fa45b8b0831246c5b2fde85e78124b88: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 11:40:29.234627 8851 raft_consensus.cc:348] T 4f5c75fa3f3b46519c60190b51d4a568 P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:29.234712 8851 raft_consensus.cc:374] T 4f5c75fa3f3b46519c60190b51d4a568 P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:29.234732 8851 raft_consensus.cc:729] T 4f5c75fa3f3b46519c60190b51d4a568 P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa45b8b0831246c5b2fde85e78124b88, State: Initialized, Role: FOLLOWER
I20260926 11:40:29.234870 8851 consensus_queue.cc:260] T 4f5c75fa3f3b46519c60190b51d4a568 P fa45b8b0831246c5b2fde85e78124b88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:29.235109 8851 ts_tablet_manager.cc:1428] T 4f5c75fa3f3b46519c60190b51d4a568 P fa45b8b0831246c5b2fde85e78124b88: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:29.346477 8805 raft_consensus.cc:482] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:40:29.346624 8805 raft_consensus.cc:504] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:29.347095 8805 leader_election.cc:288] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea9ce406518f47f18242a8d987510090 (127.6.96.2:35251), fa45b8b0831246c5b2fde85e78124b88 (127.6.96.3:36655)
I20260926 11:40:29.347537 8696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f5c75fa3f3b46519c60190b51d4a568" candidate_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa45b8b0831246c5b2fde85e78124b88" is_pre_election: true
I20260926 11:40:29.347550 8556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f5c75fa3f3b46519c60190b51d4a568" candidate_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea9ce406518f47f18242a8d987510090" is_pre_election: true
I20260926 11:40:29.347689 8696 raft_consensus.cc:2454] T 4f5c75fa3f3b46519c60190b51d4a568 P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a06f0913f7745efa75ccfde9eaf9c8d in term 0.
I20260926 11:40:29.347710 8556 raft_consensus.cc:2454] T 4f5c75fa3f3b46519c60190b51d4a568 P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a06f0913f7745efa75ccfde9eaf9c8d in term 0.
I20260926 11:40:29.347976 8347 leader_election.cc:302] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a06f0913f7745efa75ccfde9eaf9c8d, fa45b8b0831246c5b2fde85e78124b88; no voters:
I20260926 11:40:29.348137 8805 raft_consensus.cc:2781] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:40:29.348197 8805 raft_consensus.cc:482] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:40:29.348239 8805 raft_consensus.cc:3037] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:29.349150 8805 raft_consensus.cc:504] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:29.349447 8805 leader_election.cc:288] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d [CANDIDATE]: Term 1 election: Requested vote from peers ea9ce406518f47f18242a8d987510090 (127.6.96.2:35251), fa45b8b0831246c5b2fde85e78124b88 (127.6.96.3:36655)
I20260926 11:40:29.349750 8696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f5c75fa3f3b46519c60190b51d4a568" candidate_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa45b8b0831246c5b2fde85e78124b88"
I20260926 11:40:29.349756 8556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f5c75fa3f3b46519c60190b51d4a568" candidate_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea9ce406518f47f18242a8d987510090"
I20260926 11:40:29.349843 8556 raft_consensus.cc:3037] T 4f5c75fa3f3b46519c60190b51d4a568 P ea9ce406518f47f18242a8d987510090 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:29.349872 8696 raft_consensus.cc:3037] T 4f5c75fa3f3b46519c60190b51d4a568 P fa45b8b0831246c5b2fde85e78124b88 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:29.350666 8556 raft_consensus.cc:2454] T 4f5c75fa3f3b46519c60190b51d4a568 P ea9ce406518f47f18242a8d987510090 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a06f0913f7745efa75ccfde9eaf9c8d in term 1.
I20260926 11:40:29.350719 8696 raft_consensus.cc:2454] T 4f5c75fa3f3b46519c60190b51d4a568 P fa45b8b0831246c5b2fde85e78124b88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a06f0913f7745efa75ccfde9eaf9c8d in term 1.
I20260926 11:40:29.350987 8347 leader_election.cc:302] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a06f0913f7745efa75ccfde9eaf9c8d, fa45b8b0831246c5b2fde85e78124b88; no voters:
I20260926 11:40:29.351171 8805 raft_consensus.cc:2781] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:29.351258 8805 raft_consensus.cc:686] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d [term 1 LEADER]: Becoming Leader. State: Replica: 0a06f0913f7745efa75ccfde9eaf9c8d, State: Running, Role: LEADER
I20260926 11:40:29.351400 8805 consensus_queue.cc:237] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } }
I20260926 11:40:29.352878 8257 catalog_manager.cc:5495] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a06f0913f7745efa75ccfde9eaf9c8d (127.6.96.1). New cstate: current_term: 1 leader_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a06f0913f7745efa75ccfde9eaf9c8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 34477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260926 11:40:29.390066 8696 raft_consensus.cc:1261] T 4f5c75fa3f3b46519c60190b51d4a568 P fa45b8b0831246c5b2fde85e78124b88 [term 1 FOLLOWER]: Refusing update from remote peer 0a06f0913f7745efa75ccfde9eaf9c8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:29.390336 8556 raft_consensus.cc:1261] T 4f5c75fa3f3b46519c60190b51d4a568 P ea9ce406518f47f18242a8d987510090 [term 1 FOLLOWER]: Refusing update from remote peer 0a06f0913f7745efa75ccfde9eaf9c8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:29.390414 8863 consensus_queue.cc:1035] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa45b8b0831246c5b2fde85e78124b88" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:29.390743 8821 consensus_queue.cc:1035] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea9ce406518f47f18242a8d987510090" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 11:40:29.401409 8819 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 11:40:30.947427 8259 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:30.947592 8259 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:30.950008 8259 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a0b52746549b43fe83890e93abf22be8: Sending DeleteTablet for 3 replicas of tablet 4f5c75fa3f3b46519c60190b51d4a568
I20260926 11:40:30.950498 8396 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f5c75fa3f3b46519c60190b51d4a568 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:30 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:34848
I20260926 11:40:30.950580 8536 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f5c75fa3f3b46519c60190b51d4a568 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:30 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:45216
I20260926 11:40:30.950891 8676 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f5c75fa3f3b46519c60190b51d4a568 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:30 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:45928
I20260926 11:40:30.950919 8890 tablet_replica.cc:330] T 4f5c75fa3f3b46519c60190b51d4a568 P ea9ce406518f47f18242a8d987510090: stopping tablet replica
I20260926 11:40:30.950950 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 8331
I20260926 11:40:30.950819 8889 tablet_replica.cc:330] T 4f5c75fa3f3b46519c60190b51d4a568 P 0a06f0913f7745efa75ccfde9eaf9c8d: stopping tablet replica
I20260926 11:40:30.951732 8890 raft_consensus.cc:2229] T 4f5c75fa3f3b46519c60190b51d4a568 P ea9ce406518f47f18242a8d987510090 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:30.951854 8890 raft_consensus.cc:2258] T 4f5c75fa3f3b46519c60190b51d4a568 P ea9ce406518f47f18242a8d987510090 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:30.951913 8891 tablet_replica.cc:330] T 4f5c75fa3f3b46519c60190b51d4a568 P fa45b8b0831246c5b2fde85e78124b88: stopping tablet replica
I20260926 11:40:30.952085 8891 raft_consensus.cc:2229] T 4f5c75fa3f3b46519c60190b51d4a568 P fa45b8b0831246c5b2fde85e78124b88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:30.952224 8891 raft_consensus.cc:2258] T 4f5c75fa3f3b46519c60190b51d4a568 P fa45b8b0831246c5b2fde85e78124b88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:30.952425 8890 ts_tablet_manager.cc:1905] T 4f5c75fa3f3b46519c60190b51d4a568 P ea9ce406518f47f18242a8d987510090: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:30.952762 8891 ts_tablet_manager.cc:1905] T 4f5c75fa3f3b46519c60190b51d4a568 P fa45b8b0831246c5b2fde85e78124b88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:30.954509 8890 ts_tablet_manager.cc:1918] T 4f5c75fa3f3b46519c60190b51d4a568 P ea9ce406518f47f18242a8d987510090: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 11:40:30.954572 8890 log.cc:1196] T 4f5c75fa3f3b46519c60190b51d4a568 P ea9ce406518f47f18242a8d987510090: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-1/wal/wals/4f5c75fa3f3b46519c60190b51d4a568
I20260926 11:40:30.954768 8890 ts_tablet_manager.cc:1939] T 4f5c75fa3f3b46519c60190b51d4a568 P ea9ce406518f47f18242a8d987510090: Deleting consensus metadata
I20260926 11:40:30.955211 8891 ts_tablet_manager.cc:1918] T 4f5c75fa3f3b46519c60190b51d4a568 P fa45b8b0831246c5b2fde85e78124b88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 11:40:30.955312 8891 log.cc:1196] T 4f5c75fa3f3b46519c60190b51d4a568 P fa45b8b0831246c5b2fde85e78124b88: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/4f5c75fa3f3b46519c60190b51d4a568
I20260926 11:40:30.955389 8246 catalog_manager.cc:4841] TS ea9ce406518f47f18242a8d987510090 (127.6.96.2:35251): tablet 4f5c75fa3f3b46519c60190b51d4a568 (table test-table [id=ab8383631a964bf29e895eed611689c8]) successfully deleted
I20260926 11:40:30.955564 8891 ts_tablet_manager.cc:1939] T 4f5c75fa3f3b46519c60190b51d4a568 P fa45b8b0831246c5b2fde85e78124b88: Deleting consensus metadata
I20260926 11:40:30.956135 8243 catalog_manager.cc:4841] TS fa45b8b0831246c5b2fde85e78124b88 (127.6.96.3:36655): tablet 4f5c75fa3f3b46519c60190b51d4a568 (table test-table [id=ab8383631a964bf29e895eed611689c8]) successfully deleted
I20260926 11:40:30.959045 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 8471
W20260926 11:40:30.959106 8244 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.96.1:34477 (error 108)
I20260926 11:40:30.960875 8893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:30.960128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:34477 (local address 127.0.0.1:34890)
0926 11:40:30.960729 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:30.960792 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.1:34477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":78,"threads_started":1}
W20260926 11:40:30.961015 8244 catalog_manager.cc:4584] TS 0a06f0913f7745efa75ccfde9eaf9c8d (127.6.96.1:34477): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f5c75fa3f3b46519c60190b51d4a568: Network error: Client connection negotiation failed: client connection to 127.6.96.1:34477: connect: Connection refused (error 111)
I20260926 11:40:30.964001 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 8611
I20260926 11:40:30.968641 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 8227
2026-09-26T11:40:30Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6602 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:40:31 dist-test-slave-xxxx krb5kdc[8896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:40:31 dist-test-slave-xxxx krb5kdc[8896](info): set up 2 sockets
Sep 26 11:40:31 dist-test-slave-xxxx krb5kdc[8896](info): commencing operation
krb5kdc: starting...
W20260926 11:40:31.930572 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.941s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:40:32 dist-test-slave-xxxx krb5kdc[8896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:40:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:40:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:40:32.043208 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:38731
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:38731
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:32.148818 8909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:32.149080 8909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:32.149135 8909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:32.153132 8909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:40:32.153189 8909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:32.153215 8909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:32.153234 8909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:40:32.153255 8909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:40:32.158423 8909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:38731
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:38731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:32.159444 8909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:32.160416 8909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:32.168834 8917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:32.168826 8914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:32.168826 8915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:32.168974 8909 server_base.cc:1034] running on GCE node
I20260926 11:40:32.169724 8909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:32.170799 8909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:32.172005 8909 hybrid_clock.cc:648] HybridClock initialized: now 1790422832171984 us; error 33 us; skew 500 ppm
Sep 26 11:40:32 dist-test-slave-xxxx krb5kdc[8896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422832, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:32.180661 8909 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:40:32.181684 8909 webserver.cc:460] Webserver started at http://127.6.96.62:35671/ using document root <none> and password file <none>
I20260926 11:40:32.182225 8909 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:32.182289 8909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:32.182482 8909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:32.184546 8909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "d075fd60fddb40d1a3c11dfd7702edf6"
format_stamp: "Formatted at 2026-09-26 11:40:32 on dist-test-slave-xxxx"
server_key: "9d3dab7f08143a22238a5e403eefae7d"
server_key_iv: "83faf78e9b6aa0c52aacb6c7cd50b920"
server_key_version: "encryptionkey@0"
I20260926 11:40:32.185073 8909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "d075fd60fddb40d1a3c11dfd7702edf6"
format_stamp: "Formatted at 2026-09-26 11:40:32 on dist-test-slave-xxxx"
server_key: "9d3dab7f08143a22238a5e403eefae7d"
server_key_iv: "83faf78e9b6aa0c52aacb6c7cd50b920"
server_key_version: "encryptionkey@0"
I20260926 11:40:32.189505 8909 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 11:40:32.191826 8924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:32.193063 8909 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:40:32.193185 8909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "d075fd60fddb40d1a3c11dfd7702edf6"
format_stamp: "Formatted at 2026-09-26 11:40:32 on dist-test-slave-xxxx"
server_key: "9d3dab7f08143a22238a5e403eefae7d"
server_key_iv: "83faf78e9b6aa0c52aacb6c7cd50b920"
server_key_version: "encryptionkey@0"
I20260926 11:40:32.193281 8909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:32.210312 8909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:32.213107 8909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:32.213277 8909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:32.219625 8909 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:38731
I20260926 11:40:32.219662 8976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:38731 every 8 connection(s)
I20260926 11:40:32.220528 8909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:40:32.223270 8977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:32.228665 8977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d075fd60fddb40d1a3c11dfd7702edf6: Bootstrap starting.
I20260926 11:40:32.229187 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 8909
I20260926 11:40:32.229274 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:40:32.229574 6528 external_mini_cluster.cc:1428] Setting key b7178155223e100809a0746a14c58457
I20260926 11:40:32.231449 8977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d075fd60fddb40d1a3c11dfd7702edf6: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:32.232159 8977 log.cc:824] T 00000000000000000000000000000000 P d075fd60fddb40d1a3c11dfd7702edf6: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:32.233947 8977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d075fd60fddb40d1a3c11dfd7702edf6: No bootstrap required, opened a new log
I20260926 11:40:32.237323 8977 raft_consensus.cc:348] T 00000000000000000000000000000000 P d075fd60fddb40d1a3c11dfd7702edf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d075fd60fddb40d1a3c11dfd7702edf6" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 38731 } }
I20260926 11:40:32.237564 8977 raft_consensus.cc:374] T 00000000000000000000000000000000 P d075fd60fddb40d1a3c11dfd7702edf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:32.237629 8977 raft_consensus.cc:729] T 00000000000000000000000000000000 P d075fd60fddb40d1a3c11dfd7702edf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d075fd60fddb40d1a3c11dfd7702edf6, State: Initialized, Role: FOLLOWER
Sep 26 11:40:32 dist-test-slave-xxxx krb5kdc[8896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:32.238089 8977 consensus_queue.cc:260] T 00000000000000000000000000000000 P d075fd60fddb40d1a3c11dfd7702edf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d075fd60fddb40d1a3c11dfd7702edf6" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 38731 } }
I20260926 11:40:32.238221 8977 raft_consensus.cc:388] T 00000000000000000000000000000000 P d075fd60fddb40d1a3c11dfd7702edf6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:40:32.238291 8977 raft_consensus.cc:482] T 00000000000000000000000000000000 P d075fd60fddb40d1a3c11dfd7702edf6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:40:32.238384 8977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d075fd60fddb40d1a3c11dfd7702edf6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:32.239523 8977 raft_consensus.cc:504] T 00000000000000000000000000000000 P d075fd60fddb40d1a3c11dfd7702edf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d075fd60fddb40d1a3c11dfd7702edf6" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 38731 } }
I20260926 11:40:32.239902 8977 leader_election.cc:302] T 00000000000000000000000000000000 P d075fd60fddb40d1a3c11dfd7702edf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d075fd60fddb40d1a3c11dfd7702edf6; no voters:
I20260926 11:40:32.240238 8977 leader_election.cc:288] T 00000000000000000000000000000000 P d075fd60fddb40d1a3c11dfd7702edf6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:40:32.240358 8982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d075fd60fddb40d1a3c11dfd7702edf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:32.240561 8982 raft_consensus.cc:686] T 00000000000000000000000000000000 P d075fd60fddb40d1a3c11dfd7702edf6 [term 1 LEADER]: Becoming Leader. State: Replica: d075fd60fddb40d1a3c11dfd7702edf6, State: Running, Role: LEADER
I20260926 11:40:32.240895 8982 consensus_queue.cc:237] T 00000000000000000000000000000000 P d075fd60fddb40d1a3c11dfd7702edf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d075fd60fddb40d1a3c11dfd7702edf6" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 38731 } }
I20260926 11:40:32.241597 8977 sys_catalog.cc:564] T 00000000000000000000000000000000 P d075fd60fddb40d1a3c11dfd7702edf6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:40:32.242431 8983 sys_catalog.cc:455] T 00000000000000000000000000000000 P d075fd60fddb40d1a3c11dfd7702edf6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d075fd60fddb40d1a3c11dfd7702edf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d075fd60fddb40d1a3c11dfd7702edf6" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 38731 } } }
I20260926 11:40:32.242556 8983 sys_catalog.cc:458] T 00000000000000000000000000000000 P d075fd60fddb40d1a3c11dfd7702edf6 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:32.242870 8984 sys_catalog.cc:455] T 00000000000000000000000000000000 P d075fd60fddb40d1a3c11dfd7702edf6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d075fd60fddb40d1a3c11dfd7702edf6. Latest consensus state: current_term: 1 leader_uuid: "d075fd60fddb40d1a3c11dfd7702edf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d075fd60fddb40d1a3c11dfd7702edf6" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 38731 } } }
I20260926 11:40:32.242956 8984 sys_catalog.cc:458] T 00000000000000000000000000000000 P d075fd60fddb40d1a3c11dfd7702edf6 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:32.244443 8980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:32.230946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34208 (local address 127.6.96.62:38731)
0926 11:40:32.231414 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:32.231430 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:32.231470 (+ 40us) server_negotiation.cc:407] Connection header received
0926 11:40:32.232135 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:32.232153 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:32.232580 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:32.232905 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:32.233677 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:32.234669 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:32.235540 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:32.236010 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:32.236155 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:32.238492 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:32.238498 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:32.238515 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:32.238547 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:32.240406 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:32.240932 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:32.240938 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:32.240944 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:32.241088 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:32.241384 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:32.241387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:32.241389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:32.241892 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:32.242666 (+ 774us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:32.242820 (+ 154us) server_negotiation.cc:299] Negotiation successful
0926 11:40:32.243166 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":145,"threads_started":1}
W20260926 11:40:32.246814 8998 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d075fd60fddb40d1a3c11dfd7702edf6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 11:40:32.246919 8998 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 11:40:32.246971 8991 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:40:32.247789 8991 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:40:32.253285 8991 catalog_manager.cc:1347] Generated new cluster ID: 483634fde81447d8b1a379540893a0d6
I20260926 11:40:32.253345 8991 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:40:32.282234 8991 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:40:32.283293 8991 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:40:32.289990 8991 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d075fd60fddb40d1a3c11dfd7702edf6: Generated new TSK 0
I20260926 11:40:32.290663 8991 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.96.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
I20260926 11:40:32.325100 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:38731
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:32.433760 9003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:32.434038 9003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:32.434084 9003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:32.438165 9003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:32.438239 9003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:32.438318 9003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:40:32.443806 9003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:38731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:32.444865 9003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:32.445858 9003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:32.455261 9009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:32.455371 9003 server_base.cc:1034] running on GCE node
W20260926 11:40:32.455293 9008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:32.455314 9011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:32.455876 9003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:32.456493 9003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:32.457664 9003 hybrid_clock.cc:648] HybridClock initialized: now 1790422832457640 us; error 36 us; skew 500 ppm
Sep 26 11:40:32 dist-test-slave-xxxx krb5kdc[8896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422832, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:32.460310 9003 init.cc:377] Logged in from keytab as kudu/127.6.96.1@KRBTEST.COM (short username kudu)
I20260926 11:40:32.461191 9003 webserver.cc:460] Webserver started at http://127.6.96.1:45327/ using document root <none> and password file <none>
I20260926 11:40:32.461727 9003 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:32.461789 9003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:32.461971 9003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:32.463970 9003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "b6f04e956ca649c3b0ac7baf80ddc564"
format_stamp: "Formatted at 2026-09-26 11:40:32 on dist-test-slave-xxxx"
server_key: "d9869bfbfe92a6461312e2ff4dcee460"
server_key_iv: "b0053ca830b23db3d2df7e96b3196728"
server_key_version: "encryptionkey@0"
I20260926 11:40:32.464497 9003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "b6f04e956ca649c3b0ac7baf80ddc564"
format_stamp: "Formatted at 2026-09-26 11:40:32 on dist-test-slave-xxxx"
server_key: "d9869bfbfe92a6461312e2ff4dcee460"
server_key_iv: "b0053ca830b23db3d2df7e96b3196728"
server_key_version: "encryptionkey@0"
I20260926 11:40:32.468276 9003 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 11:40:32.470700 9018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:32.471984 9003 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:40:32.472100 9003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "b6f04e956ca649c3b0ac7baf80ddc564"
format_stamp: "Formatted at 2026-09-26 11:40:32 on dist-test-slave-xxxx"
server_key: "d9869bfbfe92a6461312e2ff4dcee460"
server_key_iv: "b0053ca830b23db3d2df7e96b3196728"
server_key_version: "encryptionkey@0"
I20260926 11:40:32.472198 9003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:32.487460 9003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:32.491089 9003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:32.491264 9003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:32.491845 9003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:32.492658 9003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:32.492704 9003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:32.492756 9003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:32.492781 9003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:32.501147 9003 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:35823
I20260926 11:40:32.501174 9131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:35823 every 8 connection(s)
I20260926 11:40:32.502010 9003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:40:32.511142 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 9003
Sep 26 11:40:32 dist-test-slave-xxxx krb5kdc[8896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422832, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:32.511238 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:40:32.511478 6528 external_mini_cluster.cc:1428] Setting key f3acb1d1d4b88c6c3938c8d567e4ce4a
I20260926 11:40:32.515026 8980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:32.503891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:36349 (local address 127.6.96.62:38731)
0926 11:40:32.504061 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:32.504065 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:32.504849 (+ 784us) server_negotiation.cc:407] Connection header received
0926 11:40:32.506151 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:32.506154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:32.506199 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:32.506275 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:32.507438 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:32.507980 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:32.509109 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:32.509327 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:32.509396 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:32.511779 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:32.511782 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:32.511784 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:32.511806 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:32.513120 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:32.513604 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:32.513607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:32.513608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:32.513680 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:32.513985 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:32.513988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:32.513990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:32.514165 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:32.514247 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:32.514725 (+ 478us) server_negotiation.cc:299] Negotiation successful
0926 11:40:32.514871 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 11:40:32.515884 9134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:32.504256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:38731 (local address 127.6.96.1:36349)
0926 11:40:32.504694 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:32.504737 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:32.505928 (+ 1191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:32.506423 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:32.506430 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:32.506772 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:32.507314 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:32.507328 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:32.508107 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:32.508111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:32.509000 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:32.509020 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:32.509482 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:32.509486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:32.509648 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:32.510209 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:32.510225 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:32.511619 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:32.513222 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:32.513227 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:32.513243 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:32.513484 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:32.513764 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:32.513767 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:32.513769 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:32.513902 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:32.514257 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:32.514264 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:32.514502 (+ 238us) client_negotiation.cc:769] Sending connection context
0926 11:40:32.514668 (+ 166us) client_negotiation.cc:240] Negotiation successful
0926 11:40:32.515019 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":113,"threads_started":1}
I20260926 11:40:32.517099 9132 heartbeater.cc:347] Connected to a master server at 127.6.96.62:38731
I20260926 11:40:32.517315 9132 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:32.517767 9132 heartbeater.cc:511] Master 127.6.96.62:38731 requested a full tablet report, sending...
I20260926 11:40:32.519218 8941 ts_manager.cc:194] Registered new tserver with Master: b6f04e956ca649c3b0ac7baf80ddc564 (127.6.96.1:35823)
WARNING: no policy specified for kudu/127.6.96.2@KRBTEST.COM; defaulting to no policy
I20260926 11:40:32.520313 8941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.1@KRBTEST.COM'} at 127.6.96.1:36349
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
I20260926 11:40:32.538033 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:38731
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:32.645345 9137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:32.645637 9137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:32.645680 9137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:32.649793 9137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:32.649871 9137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:32.649931 9137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:40:32.655305 9137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:38731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:32.656347 9137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:32.657323 9137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:32.666620 9143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:32.666621 9145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:32.666622 9142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:32.666931 9137 server_base.cc:1034] running on GCE node
I20260926 11:40:32.667450 9137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:32.668174 9137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:32.669343 9137 hybrid_clock.cc:648] HybridClock initialized: now 1790422832669322 us; error 31 us; skew 500 ppm
Sep 26 11:40:32 dist-test-slave-xxxx krb5kdc[8896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422832, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:32.672123 9137 init.cc:377] Logged in from keytab as kudu/127.6.96.2@KRBTEST.COM (short username kudu)
I20260926 11:40:32.673036 9137 webserver.cc:460] Webserver started at http://127.6.96.2:36855/ using document root <none> and password file <none>
I20260926 11:40:32.673607 9137 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:32.673673 9137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:32.673873 9137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:32.676087 9137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "0bf4ebd5e92b414b9e10d11c2b82cd66"
format_stamp: "Formatted at 2026-09-26 11:40:32 on dist-test-slave-xxxx"
server_key: "c2be40a0a8c8ce06edb965a4a58c67ad"
server_key_iv: "114623c04e0686345a443e7b6a06922f"
server_key_version: "encryptionkey@0"
I20260926 11:40:32.676631 9137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "0bf4ebd5e92b414b9e10d11c2b82cd66"
format_stamp: "Formatted at 2026-09-26 11:40:32 on dist-test-slave-xxxx"
server_key: "c2be40a0a8c8ce06edb965a4a58c67ad"
server_key_iv: "114623c04e0686345a443e7b6a06922f"
server_key_version: "encryptionkey@0"
I20260926 11:40:32.680320 9137 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 11:40:32.682798 9152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:32.684115 9137 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:40:32.684211 9137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "0bf4ebd5e92b414b9e10d11c2b82cd66"
format_stamp: "Formatted at 2026-09-26 11:40:32 on dist-test-slave-xxxx"
server_key: "c2be40a0a8c8ce06edb965a4a58c67ad"
server_key_iv: "114623c04e0686345a443e7b6a06922f"
server_key_version: "encryptionkey@0"
I20260926 11:40:32.684298 9137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:32.710955 9137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:32.713614 9137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:32.713760 9137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:32.714274 9137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:32.715078 9137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:32.715124 9137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:32.715163 9137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:32.715183 9137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:32.725081 9137 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:34677
I20260926 11:40:32.725427 9265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:34677 every 8 connection(s)
I20260926 11:40:32.726177 9137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:40:32.733886 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 9137
I20260926 11:40:32.733979 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:40:32.734234 6528 external_mini_cluster.cc:1428] Setting key e8946a8a82e2e42cc7934f8e8fa64d87
Sep 26 11:40:32 dist-test-slave-xxxx krb5kdc[8896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422832, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:32.739283 8980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:32.728082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:35637 (local address 127.6.96.62:38731)
0926 11:40:32.728202 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:32.728205 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:32.729015 (+ 810us) server_negotiation.cc:407] Connection header received
0926 11:40:32.730325 (+ 1310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:32.730329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:32.730384 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:32.730455 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:32.731673 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:32.732187 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:32.733284 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:32.733505 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:32.733600 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:32.735946 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:32.735950 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:32.735952 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:32.735979 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:32.737332 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:32.737871 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:32.737874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:32.737875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:32.737948 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:32.738267 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:32.738270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:32.738271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:32.738429 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:32.738503 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:32.739075 (+ 572us) server_negotiation.cc:299] Negotiation successful
0926 11:40:32.739169 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 11:40:32.740391 9268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:32.728442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:38731 (local address 127.6.96.2:35637)
0926 11:40:32.728874 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:32.728914 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:32.730123 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:32.730619 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:32.730626 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:32.731076 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:32.731535 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:32.731548 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:32.732301 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:32.732304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:32.733169 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:32.733177 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:32.733637 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:32.733640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:32.733806 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:32.734324 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:32.734341 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:32.735788 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:32.737437 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:32.737442 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:32.737457 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:32.737778 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:32.738025 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:32.738029 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:32.738031 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:32.738194 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:32.738510 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:32.738516 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:32.738851 (+ 335us) client_negotiation.cc:769] Sending connection context
0926 11:40:32.739044 (+ 193us) client_negotiation.cc:240] Negotiation successful
0926 11:40:32.739364 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":128,"threads_started":1}
I20260926 11:40:32.741477 9266 heartbeater.cc:347] Connected to a master server at 127.6.96.62:38731
I20260926 11:40:32.741710 9266 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:32.742197 9266 heartbeater.cc:511] Master 127.6.96.62:38731 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.96.3@KRBTEST.COM; defaulting to no policy
I20260926 11:40:32.743304 8941 ts_manager.cc:194] Registered new tserver with Master: 0bf4ebd5e92b414b9e10d11c2b82cd66 (127.6.96.2:34677)
I20260926 11:40:32.743896 8941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.2@KRBTEST.COM'} at 127.6.96.2:35637
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
I20260926 11:40:32.761293 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:38731
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:32.867856 9271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:32.868115 9271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:32.868172 9271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:32.872198 9271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:32.872248 9271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:32.872327 9271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:40:32.877466 9271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:38731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:32.878459 9271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:32.879444 9271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:32.888401 9276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:32.888428 9279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:32.888471 9271 server_base.cc:1034] running on GCE node
W20260926 11:40:32.888401 9277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:32.889012 9271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:32.889618 9271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:32.890791 9271 hybrid_clock.cc:648] HybridClock initialized: now 1790422832890770 us; error 31 us; skew 500 ppm
Sep 26 11:40:32 dist-test-slave-xxxx krb5kdc[8896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422832, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:32.893347 9271 init.cc:377] Logged in from keytab as kudu/127.6.96.3@KRBTEST.COM (short username kudu)
I20260926 11:40:32.894172 9271 webserver.cc:460] Webserver started at http://127.6.96.3:37995/ using document root <none> and password file <none>
I20260926 11:40:32.894686 9271 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:32.894747 9271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:32.894969 9271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:32.897058 9271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "8bf327231174497ab8a76d5b7b8ce84e"
format_stamp: "Formatted at 2026-09-26 11:40:32 on dist-test-slave-xxxx"
server_key: "e8e6ef7fbeb64e3507f6d893cf85def5"
server_key_iv: "6fd8b006e76676dccd39297f5669f8b1"
server_key_version: "encryptionkey@0"
I20260926 11:40:32.897614 9271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "8bf327231174497ab8a76d5b7b8ce84e"
format_stamp: "Formatted at 2026-09-26 11:40:32 on dist-test-slave-xxxx"
server_key: "e8e6ef7fbeb64e3507f6d893cf85def5"
server_key_iv: "6fd8b006e76676dccd39297f5669f8b1"
server_key_version: "encryptionkey@0"
I20260926 11:40:32.901288 9271 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 11:40:32.903611 9286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:32.904776 9271 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:40:32.904871 9271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "8bf327231174497ab8a76d5b7b8ce84e"
format_stamp: "Formatted at 2026-09-26 11:40:32 on dist-test-slave-xxxx"
server_key: "e8e6ef7fbeb64e3507f6d893cf85def5"
server_key_iv: "6fd8b006e76676dccd39297f5669f8b1"
server_key_version: "encryptionkey@0"
I20260926 11:40:32.904970 9271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:32.939100 9271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:32.942322 9271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:32.942503 9271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:32.943140 9271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:32.943939 9271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:32.943984 9271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:32.944023 9271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:32.944051 9271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:32.952204 9271 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:36229
I20260926 11:40:32.952234 9399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:36229 every 8 connection(s)
I20260926 11:40:32.953049 9271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:40:32.957047 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 9271
I20260926 11:40:32.957145 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:40:32.957437 6528 external_mini_cluster.cc:1428] Setting key c2ccc555949c641f2ddcf2b9e5aff4df
Sep 26 11:40:32 dist-test-slave-xxxx krb5kdc[8896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422832, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:32.965981 8980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:32.954866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:54047 (local address 127.6.96.62:38731)
0926 11:40:32.954998 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:32.955002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:32.955878 (+ 876us) server_negotiation.cc:407] Connection header received
0926 11:40:32.957197 (+ 1319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:32.957201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:32.957255 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:32.957349 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:32.958667 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:32.959278 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:32.960436 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:32.960659 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:32.960731 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:32.962960 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:32.962963 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:32.962965 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:32.962987 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:32.964229 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:32.964679 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:32.964681 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:32.964682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:32.964756 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:32.965002 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:32.965004 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:32.965006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:32.965167 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:32.965253 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:32.965761 (+ 508us) server_negotiation.cc:299] Negotiation successful
0926 11:40:32.965865 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 11:40:32.966799 9402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:32.955244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:38731 (local address 127.6.96.3:54047)
0926 11:40:32.955723 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:32.955768 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:32.956982 (+ 1214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:32.957515 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:32.957524 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:32.957976 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:32.958528 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:32.958555 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:32.959382 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:32.959386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:32.960325 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:32.960335 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:32.960768 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:32.960771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:32.960908 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:32.961388 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:32.961403 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:32.962763 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:32.964339 (+ 1576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:32.964343 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:32.964355 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:32.964600 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:32.964823 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:32.964825 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:32.964826 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:32.964946 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:32.965253 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:32.965258 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:32.965496 (+ 238us) client_negotiation.cc:769] Sending connection context
0926 11:40:32.965702 (+ 206us) client_negotiation.cc:240] Negotiation successful
0926 11:40:32.966008 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":111,"threads_started":1}
I20260926 11:40:32.967865 9400 heartbeater.cc:347] Connected to a master server at 127.6.96.62:38731
I20260926 11:40:32.968080 9400 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:32.968537 9400 heartbeater.cc:511] Master 127.6.96.62:38731 requested a full tablet report, sending...
I20260926 11:40:32.969456 8941 ts_manager.cc:194] Registered new tserver with Master: 8bf327231174497ab8a76d5b7b8ce84e (127.6.96.3:36229)
I20260926 11:40:32.970083 8941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.3@KRBTEST.COM'} at 127.6.96.3:54047
I20260926 11:40:32.971143 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 11:40:32.982538 8980 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:32.981244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34212 (local address 127.6.96.62:38731)
0926 11:40:32.981383 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:32.981388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:32.981525 (+ 137us) server_negotiation.cc:407] Connection header received
0926 11:40:32.981669 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:32.981672 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:32.981713 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:32.981778 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:32.982408 (+ 630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34212: BlockingRecv error: recv got EOF from 127.0.0.1:34212 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 11:40:32.983579 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 9003
I20260926 11:40:32.988189 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 9137
I20260926 11:40:32.992677 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 9271
I20260926 11:40:32.996869 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 8909
2026-09-26T11:40:32Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2027 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:40:33 dist-test-slave-xxxx krb5kdc[9412](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:40:33 dist-test-slave-xxxx krb5kdc[9412](info): set up 2 sockets
Sep 26 11:40:33 dist-test-slave-xxxx krb5kdc[9412](info): commencing operation
krb5kdc: starting...
W20260926 11:40:33.950639 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:40:34 dist-test-slave-xxxx krb5kdc[9412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:40:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:40:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:40:34.064139 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:35119
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:34485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:35119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:34.173343 9425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:34.173588 9425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:34.173626 9425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:34.177558 9425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:40:34.177614 9425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:34.177632 9425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:34.177654 9425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:40:34.177668 9425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:40:34.182701 9425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:34485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:35119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:35119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:34.183676 9425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:34.184571 9425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:34.192659 9433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:34.192793 9425 server_base.cc:1034] running on GCE node
W20260926 11:40:34.192670 9430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:34.192682 9431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:34.193656 9425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:34.195060 9425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:34.196226 9425 hybrid_clock.cc:648] HybridClock initialized: now 1790422834196206 us; error 38 us; skew 500 ppm
Sep 26 11:40:34 dist-test-slave-xxxx krb5kdc[9412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422834, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:34.198866 9425 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:40:34.199759 9425 webserver.cc:460] Webserver started at http://127.6.96.62:40869/ using document root <none> and password file <none>
I20260926 11:40:34.200295 9425 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:34.200353 9425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:34.200552 9425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:34.202507 9425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "b6aee6cf75ab45b5b5bab8c941cafbc5"
format_stamp: "Formatted at 2026-09-26 11:40:34 on dist-test-slave-xxxx"
server_key: "4758e729193df1135fd7d98b2bf48c86"
server_key_iv: "28faa15dfcb2a82be99042a90fbe0968"
server_key_version: "encryptionkey@0"
I20260926 11:40:34.203049 9425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "b6aee6cf75ab45b5b5bab8c941cafbc5"
format_stamp: "Formatted at 2026-09-26 11:40:34 on dist-test-slave-xxxx"
server_key: "4758e729193df1135fd7d98b2bf48c86"
server_key_iv: "28faa15dfcb2a82be99042a90fbe0968"
server_key_version: "encryptionkey@0"
I20260926 11:40:34.206753 9425 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260926 11:40:34.209213 9440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:34.210443 9425 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:34.210556 9425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "b6aee6cf75ab45b5b5bab8c941cafbc5"
format_stamp: "Formatted at 2026-09-26 11:40:34 on dist-test-slave-xxxx"
server_key: "4758e729193df1135fd7d98b2bf48c86"
server_key_iv: "28faa15dfcb2a82be99042a90fbe0968"
server_key_version: "encryptionkey@0"
I20260926 11:40:34.210667 9425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:34.220345 9425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:34.223163 9425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:34.223333 9425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:34.230021 9425 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:35119
I20260926 11:40:34.230265 9492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:35119 every 8 connection(s)
I20260926 11:40:34.231048 9425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:40:34.233901 9493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:34.239315 9493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6aee6cf75ab45b5b5bab8c941cafbc5: Bootstrap starting.
I20260926 11:40:34.239650 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 9425
I20260926 11:40:34.239753 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:40:34.239976 6528 external_mini_cluster.cc:1428] Setting key 6d72cd033317db3975fdf3a101dea6ac
I20260926 11:40:34.241828 9493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6aee6cf75ab45b5b5bab8c941cafbc5: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:34.242555 9493 log.cc:824] T 00000000000000000000000000000000 P b6aee6cf75ab45b5b5bab8c941cafbc5: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:34.244724 9493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6aee6cf75ab45b5b5bab8c941cafbc5: No bootstrap required, opened a new log
Sep 26 11:40:34 dist-test-slave-xxxx krb5kdc[9412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:34.248847 9493 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6aee6cf75ab45b5b5bab8c941cafbc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6aee6cf75ab45b5b5bab8c941cafbc5" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 35119 } }
I20260926 11:40:34.249121 9493 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6aee6cf75ab45b5b5bab8c941cafbc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:34.249190 9493 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6aee6cf75ab45b5b5bab8c941cafbc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6aee6cf75ab45b5b5bab8c941cafbc5, State: Initialized, Role: FOLLOWER
I20260926 11:40:34.249668 9493 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6aee6cf75ab45b5b5bab8c941cafbc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6aee6cf75ab45b5b5bab8c941cafbc5" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 35119 } }
I20260926 11:40:34.249804 9493 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6aee6cf75ab45b5b5bab8c941cafbc5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:40:34.249869 9493 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6aee6cf75ab45b5b5bab8c941cafbc5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:40:34.249961 9493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6aee6cf75ab45b5b5bab8c941cafbc5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:34.250785 9493 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6aee6cf75ab45b5b5bab8c941cafbc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6aee6cf75ab45b5b5bab8c941cafbc5" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 35119 } }
I20260926 11:40:34.251130 9493 leader_election.cc:302] T 00000000000000000000000000000000 P b6aee6cf75ab45b5b5bab8c941cafbc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6aee6cf75ab45b5b5bab8c941cafbc5; no voters:
I20260926 11:40:34.251458 9493 leader_election.cc:288] T 00000000000000000000000000000000 P b6aee6cf75ab45b5b5bab8c941cafbc5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:40:34.251521 9498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6aee6cf75ab45b5b5bab8c941cafbc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:34.251741 9498 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6aee6cf75ab45b5b5bab8c941cafbc5 [term 1 LEADER]: Becoming Leader. State: Replica: b6aee6cf75ab45b5b5bab8c941cafbc5, State: Running, Role: LEADER
I20260926 11:40:34.252053 9498 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6aee6cf75ab45b5b5bab8c941cafbc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6aee6cf75ab45b5b5bab8c941cafbc5" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 35119 } }
I20260926 11:40:34.252795 9493 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6aee6cf75ab45b5b5bab8c941cafbc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:40:34.254491 9496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:34.241229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55190 (local address 127.6.96.62:35119)
0926 11:40:34.241668 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:34.241683 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:34.241726 (+ 43us) server_negotiation.cc:407] Connection header received
0926 11:40:34.242390 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:34.242409 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:34.242790 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:34.243201 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:34.244033 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:34.245042 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:34.246176 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:34.246636 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:34.246754 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:34.249497 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:34.249505 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:34.249522 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:34.249563 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:34.251242 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:34.251736 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:34.251743 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:34.251750 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:34.251890 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:34.252207 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:34.252211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:34.252213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:34.252702 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:34.252902 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:34.253185 (+ 283us) server_negotiation.cc:299] Negotiation successful
0926 11:40:34.253473 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":121,"threads_started":1}
I20260926 11:40:34.255385 9500 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6aee6cf75ab45b5b5bab8c941cafbc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6aee6cf75ab45b5b5bab8c941cafbc5. Latest consensus state: current_term: 1 leader_uuid: "b6aee6cf75ab45b5b5bab8c941cafbc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6aee6cf75ab45b5b5bab8c941cafbc5" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 35119 } } }
I20260926 11:40:34.255501 9500 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6aee6cf75ab45b5b5bab8c941cafbc5 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:34.255786 9499 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6aee6cf75ab45b5b5bab8c941cafbc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6aee6cf75ab45b5b5bab8c941cafbc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6aee6cf75ab45b5b5bab8c941cafbc5" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 35119 } } }
I20260926 11:40:34.255868 9499 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6aee6cf75ab45b5b5bab8c941cafbc5 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:34.257378 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260926 11:40:34.257951 9513 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b6aee6cf75ab45b5b5bab8c941cafbc5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 11:40:34.258026 9513 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 11:40:34.258081 9514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:40:34.258931 9514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:40:34.264788 9514 catalog_manager.cc:1347] Generated new cluster ID: ebf600db8163481ebe8f55da2d1ebd2e
I20260926 11:40:34.264853 9514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:40:34.272338 9496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:34.268066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55202 (local address 127.6.96.62:35119)
0926 11:40:34.268178 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:34.268183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:34.268410 (+ 227us) server_negotiation.cc:407] Connection header received
0926 11:40:34.268534 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:34.268537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:34.268579 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:34.268641 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:34.269126 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:34.269636 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:34.270441 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:34.270689 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:34.270773 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:34.271525 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:34.271529 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:34.271531 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:40:34.271938 (+ 407us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:34.271964 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:40:34.271979 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:34.272055 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:34.272150 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 11:40:34.272207 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 11:40:34.280210 9514 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:40:34.281116 9514 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:40:34.285745 9514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6aee6cf75ab45b5b5bab8c941cafbc5: Generated new TSK 0
I20260926 11:40:34.286443 9514 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:40:34.297649 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 9425
2026-09-26T11:40:34Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1300 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:40:34 dist-test-slave-xxxx krb5kdc[9526](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:40:34 dist-test-slave-xxxx krb5kdc[9526](info): set up 2 sockets
Sep 26 11:40:34 dist-test-slave-xxxx krb5kdc[9526](info): commencing operation
krb5kdc: starting...
W20260926 11:40:35.250484 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:40:35 dist-test-slave-xxxx krb5kdc[9526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:40:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:40:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:40:35.361441 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:33507
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:37229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:33507
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:35.469164 9539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:35.469447 9539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:35.469486 9539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:35.473460 9539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:40:35.473515 9539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:35.473533 9539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:35.473548 9539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:40:35.473562 9539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:40:35.479266 9539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:37229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:33507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:33507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:35.480300 9539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:35.481259 9539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:35.489475 9545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:35.489495 9547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:35.489590 9539 server_base.cc:1034] running on GCE node
W20260926 11:40:35.489487 9544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:35.490272 9539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:35.491470 9539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:35.492645 9539 hybrid_clock.cc:648] HybridClock initialized: now 1790422835492624 us; error 34 us; skew 500 ppm
Sep 26 11:40:35 dist-test-slave-xxxx krb5kdc[9526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422835, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:35.495364 9539 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:40:35.496294 9539 webserver.cc:460] Webserver started at http://127.6.96.62:34129/ using document root <none> and password file <none>
I20260926 11:40:35.496961 9539 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:35.497036 9539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:35.497248 9539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:35.499349 9539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "37528eeb9e644a3f8ce59639206d1000"
format_stamp: "Formatted at 2026-09-26 11:40:35 on dist-test-slave-xxxx"
server_key: "66e997d2b9b834d826e99b5642d9bfee"
server_key_iv: "a8d8e276955fa1211414caaad6c3ee9c"
server_key_version: "encryptionkey@0"
I20260926 11:40:35.499883 9539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "37528eeb9e644a3f8ce59639206d1000"
format_stamp: "Formatted at 2026-09-26 11:40:35 on dist-test-slave-xxxx"
server_key: "66e997d2b9b834d826e99b5642d9bfee"
server_key_iv: "a8d8e276955fa1211414caaad6c3ee9c"
server_key_version: "encryptionkey@0"
I20260926 11:40:35.503410 9539 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:40:35.505686 9554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:35.506938 9539 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:40:35.507046 9539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "37528eeb9e644a3f8ce59639206d1000"
format_stamp: "Formatted at 2026-09-26 11:40:35 on dist-test-slave-xxxx"
server_key: "66e997d2b9b834d826e99b5642d9bfee"
server_key_iv: "a8d8e276955fa1211414caaad6c3ee9c"
server_key_version: "encryptionkey@0"
I20260926 11:40:35.507136 9539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:35.519428 9539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:35.522150 9539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:35.522316 9539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:35.528822 9539 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:33507
I20260926 11:40:35.529062 9606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:33507 every 8 connection(s)
I20260926 11:40:35.529757 9539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:40:35.532490 9607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:35.537397 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 9539
I20260926 11:40:35.537520 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:40:35.537783 6528 external_mini_cluster.cc:1428] Setting key 4cc3bdf893921ef20cc3b17c68f395c4
I20260926 11:40:35.537909 9607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37528eeb9e644a3f8ce59639206d1000: Bootstrap starting.
I20260926 11:40:35.540258 9607 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37528eeb9e644a3f8ce59639206d1000: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:35.541020 9607 log.cc:824] T 00000000000000000000000000000000 P 37528eeb9e644a3f8ce59639206d1000: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:35.542848 9607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37528eeb9e644a3f8ce59639206d1000: No bootstrap required, opened a new log
Sep 26 11:40:35 dist-test-slave-xxxx krb5kdc[9526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:35.546747 9607 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37528eeb9e644a3f8ce59639206d1000 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37528eeb9e644a3f8ce59639206d1000" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 33507 } }
I20260926 11:40:35.547003 9607 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37528eeb9e644a3f8ce59639206d1000 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:35.547060 9607 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37528eeb9e644a3f8ce59639206d1000 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37528eeb9e644a3f8ce59639206d1000, State: Initialized, Role: FOLLOWER
I20260926 11:40:35.547458 9607 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37528eeb9e644a3f8ce59639206d1000 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37528eeb9e644a3f8ce59639206d1000" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 33507 } }
I20260926 11:40:35.547569 9607 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37528eeb9e644a3f8ce59639206d1000 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:40:35.547624 9607 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37528eeb9e644a3f8ce59639206d1000 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:40:35.547720 9607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37528eeb9e644a3f8ce59639206d1000 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:35.548745 9607 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37528eeb9e644a3f8ce59639206d1000 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37528eeb9e644a3f8ce59639206d1000" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 33507 } }
I20260926 11:40:35.549086 9607 leader_election.cc:302] T 00000000000000000000000000000000 P 37528eeb9e644a3f8ce59639206d1000 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37528eeb9e644a3f8ce59639206d1000; no voters:
I20260926 11:40:35.549400 9607 leader_election.cc:288] T 00000000000000000000000000000000 P 37528eeb9e644a3f8ce59639206d1000 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:40:35.549451 9612 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37528eeb9e644a3f8ce59639206d1000 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:35.549669 9612 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37528eeb9e644a3f8ce59639206d1000 [term 1 LEADER]: Becoming Leader. State: Replica: 37528eeb9e644a3f8ce59639206d1000, State: Running, Role: LEADER
I20260926 11:40:35.550668 9607 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37528eeb9e644a3f8ce59639206d1000 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:40:35.551664 9610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:35.539049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56038 (local address 127.6.96.62:33507)
0926 11:40:35.539424 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:35.539439 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:35.539475 (+ 36us) server_negotiation.cc:407] Connection header received
0926 11:40:35.540071 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:35.540087 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:35.540425 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:35.540691 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:35.541505 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:35.542539 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:35.543673 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:35.544123 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:35.544242 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:35.546884 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:35.546889 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:35.546902 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:35.546949 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:35.548838 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:35.549216 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:35.549221 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:35.549227 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:35.549362 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:35.549653 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:35.549656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:35.549657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:35.550168 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:35.550542 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:35.550691 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 11:40:35.550985 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":104,"threads_started":1}
I20260926 11:40:35.552067 9612 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37528eeb9e644a3f8ce59639206d1000 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37528eeb9e644a3f8ce59639206d1000" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 33507 } }
I20260926 11:40:35.553720 9613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37528eeb9e644a3f8ce59639206d1000 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37528eeb9e644a3f8ce59639206d1000. Latest consensus state: current_term: 1 leader_uuid: "37528eeb9e644a3f8ce59639206d1000" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37528eeb9e644a3f8ce59639206d1000" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 33507 } } }
I20260926 11:40:35.553766 9614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37528eeb9e644a3f8ce59639206d1000 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37528eeb9e644a3f8ce59639206d1000" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37528eeb9e644a3f8ce59639206d1000" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 33507 } } }
I20260926 11:40:35.553879 9613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37528eeb9e644a3f8ce59639206d1000 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:35.553941 9614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37528eeb9e644a3f8ce59639206d1000 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:35.554273 9621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:40:35.557245 9621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:40:35.562852 9621 catalog_manager.cc:1347] Generated new cluster ID: 6b9f87e4f97d4e3b91eca91967a09b85
I20260926 11:40:35.562927 9621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:40:35.580178 9621 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:40:35.581203 9621 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:40:35.587594 9621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37528eeb9e644a3f8ce59639206d1000: Generated new TSK 0
I20260926 11:40:35.588249 9621 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.96.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
I20260926 11:40:35.617676 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:33507
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:37229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:35.732164 9633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:35.732745 9633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:35.732844 9633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:35.738951 9633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:35.739092 9633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:35.739233 9633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:40:35.747365 9633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:37229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:33507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:35.748780 9633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:35.750126 9633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:35.761188 9638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:35.761245 9639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:35.761195 9641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:35.761755 9633 server_base.cc:1034] running on GCE node
I20260926 11:40:35.762143 9633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:35.762768 9633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:35.763921 9633 hybrid_clock.cc:648] HybridClock initialized: now 1790422835763892 us; error 46 us; skew 500 ppm
Sep 26 11:40:35 dist-test-slave-xxxx krb5kdc[9526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422835, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:35.766500 9633 init.cc:377] Logged in from keytab as kudu/127.6.96.1@KRBTEST.COM (short username kudu)
I20260926 11:40:35.767383 9633 webserver.cc:460] Webserver started at http://127.6.96.1:43397/ using document root <none> and password file <none>
I20260926 11:40:35.767897 9633 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:35.767959 9633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:35.768139 9633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:35.770220 9633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "1f8b126aba6c47ad8d7b8c6b59ea525e"
format_stamp: "Formatted at 2026-09-26 11:40:35 on dist-test-slave-xxxx"
server_key: "ee67c3c68c2dc420675410de699c1ea7"
server_key_iv: "571b711280f0090f1b4886a23a48531b"
server_key_version: "encryptionkey@0"
I20260926 11:40:35.770757 9633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "1f8b126aba6c47ad8d7b8c6b59ea525e"
format_stamp: "Formatted at 2026-09-26 11:40:35 on dist-test-slave-xxxx"
server_key: "ee67c3c68c2dc420675410de699c1ea7"
server_key_iv: "571b711280f0090f1b4886a23a48531b"
server_key_version: "encryptionkey@0"
I20260926 11:40:35.774523 9633 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260926 11:40:35.777031 9648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:35.778383 9633 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:35.778496 9633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "1f8b126aba6c47ad8d7b8c6b59ea525e"
format_stamp: "Formatted at 2026-09-26 11:40:35 on dist-test-slave-xxxx"
server_key: "ee67c3c68c2dc420675410de699c1ea7"
server_key_iv: "571b711280f0090f1b4886a23a48531b"
server_key_version: "encryptionkey@0"
I20260926 11:40:35.778594 9633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:35.794430 9633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:35.797339 9633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:35.797529 9633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:35.798105 9633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:35.798936 9633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:35.798981 9633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:35.799033 9633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:35.799057 9633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:35.807220 9633 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:37869
I20260926 11:40:35.807246 9761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:37869 every 8 connection(s)
I20260926 11:40:35.808151 9633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:40:35.813791 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 9633
I20260926 11:40:35.813895 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:40:35.814142 6528 external_mini_cluster.cc:1428] Setting key c44de9eca607ee0a4d7e3af443b6348d
Sep 26 11:40:35 dist-test-slave-xxxx krb5kdc[9526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422835, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:35.821709 9610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:35.809952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:49337 (local address 127.6.96.62:33507)
0926 11:40:35.810096 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:35.810101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:35.811037 (+ 936us) server_negotiation.cc:407] Connection header received
0926 11:40:35.812462 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:35.812465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:35.812512 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:35.812586 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:35.813787 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:35.814382 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:35.815586 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:35.815823 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:35.815914 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:35.818475 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:35.818479 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:35.818482 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:35.818507 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:35.819845 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:35.820319 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:35.820323 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:35.820324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:35.820395 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:35.820689 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:35.820692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:35.820694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:35.820862 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:35.820943 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:35.821419 (+ 476us) server_negotiation.cc:299] Negotiation successful
0926 11:40:35.821537 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
WARNING: no policy specified for kudu/127.6.96.2@KRBTEST.COM; defaulting to no policy
I20260926 11:40:35.822516 9764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:35.810385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:33507 (local address 127.6.96.1:49337)
0926 11:40:35.810876 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:35.810934 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:35.812261 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:35.812736 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:35.812743 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:35.813086 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:35.813633 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:35.813654 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:35.814500 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:35.814503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:35.815466 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:35.815476 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:35.815965 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:35.815969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:35.816121 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:35.816683 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:35.816704 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:35.818322 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:35.819950 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:35.819955 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:35.819970 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:35.820231 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:35.820473 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:35.820476 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:35.820478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:35.820608 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:35.820953 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:35.820960 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:35.821207 (+ 247us) client_negotiation.cc:769] Sending connection context
0926 11:40:35.821367 (+ 160us) client_negotiation.cc:240] Negotiation successful
0926 11:40:35.821681 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":115,"threads_started":1}
I20260926 11:40:35.823635 9762 heartbeater.cc:347] Connected to a master server at 127.6.96.62:33507
I20260926 11:40:35.823860 9762 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:35.824385 9762 heartbeater.cc:511] Master 127.6.96.62:33507 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.2@KRBTEST.COM" created.
I20260926 11:40:35.825984 9571 ts_manager.cc:194] Registered new tserver with Master: 1f8b126aba6c47ad8d7b8c6b59ea525e (127.6.96.1:37869)
I20260926 11:40:35.827489 9571 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.1@KRBTEST.COM'} at 127.6.96.1:49337
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
I20260926 11:40:35.841678 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:33507
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:37229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:35.949177 9767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:35.949427 9767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:35.949469 9767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:35.953565 9767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:35.953621 9767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:35.953693 9767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:40:35.958861 9767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:37229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:33507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:35.959828 9767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:35.960788 9767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:35.969726 9773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:35.969832 9772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:35.969743 9775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:35.969975 9767 server_base.cc:1034] running on GCE node
I20260926 11:40:35.970541 9767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:35.971200 9767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:35.972361 9767 hybrid_clock.cc:648] HybridClock initialized: now 1790422835972343 us; error 34 us; skew 500 ppm
Sep 26 11:40:35 dist-test-slave-xxxx krb5kdc[9526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422835, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:35.974964 9767 init.cc:377] Logged in from keytab as kudu/127.6.96.2@KRBTEST.COM (short username kudu)
I20260926 11:40:35.975821 9767 webserver.cc:460] Webserver started at http://127.6.96.2:35307/ using document root <none> and password file <none>
I20260926 11:40:35.976315 9767 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:35.976369 9767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:35.976527 9767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:35.978437 9767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "c9b375e161684c72a35b88533fbcefb5"
format_stamp: "Formatted at 2026-09-26 11:40:35 on dist-test-slave-xxxx"
server_key: "67c1c5783d0fe1492345d8b50811a309"
server_key_iv: "e35e03fcbc5f9429703aa2712451b0c3"
server_key_version: "encryptionkey@0"
I20260926 11:40:35.978991 9767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "c9b375e161684c72a35b88533fbcefb5"
format_stamp: "Formatted at 2026-09-26 11:40:35 on dist-test-slave-xxxx"
server_key: "67c1c5783d0fe1492345d8b50811a309"
server_key_iv: "e35e03fcbc5f9429703aa2712451b0c3"
server_key_version: "encryptionkey@0"
I20260926 11:40:35.982515 9767 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:40:35.984855 9782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:35.986004 9767 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:35.986095 9767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "c9b375e161684c72a35b88533fbcefb5"
format_stamp: "Formatted at 2026-09-26 11:40:35 on dist-test-slave-xxxx"
server_key: "67c1c5783d0fe1492345d8b50811a309"
server_key_iv: "e35e03fcbc5f9429703aa2712451b0c3"
server_key_version: "encryptionkey@0"
I20260926 11:40:35.986191 9767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:35.998417 9767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:36.001169 9767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:36.001322 9767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:36.001857 9767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:36.002614 9767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:36.002657 9767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:36.002694 9767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:36.002722 9767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:36.010999 9767 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:40567
I20260926 11:40:36.011018 9895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:40567 every 8 connection(s)
I20260926 11:40:36.011857 9767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:40:36.017572 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 9767
I20260926 11:40:36.017696 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:40:36.017956 6528 external_mini_cluster.cc:1428] Setting key 4debef521725cb63096ff29f223b8923
Sep 26 11:40:36 dist-test-slave-xxxx krb5kdc[9526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422835, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:36.025425 9610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:36.013501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:33039 (local address 127.6.96.62:33507)
0926 11:40:36.013621 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:36.013624 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:36.014415 (+ 791us) server_negotiation.cc:407] Connection header received
0926 11:40:36.015766 (+ 1351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:36.015770 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:36.015825 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:36.015894 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:36.016998 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:36.017627 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:36.018799 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:36.019103 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:36.019203 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:36.021979 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:36.021982 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:36.021984 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:36.022006 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:36.023535 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:36.024005 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:36.024008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:36.024009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:36.024079 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:36.024374 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:36.024378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:36.024379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:36.024552 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:36.024634 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:36.025136 (+ 502us) server_negotiation.cc:299] Negotiation successful
0926 11:40:36.025267 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for kudu/127.6.96.3@KRBTEST.COM; defaulting to no policy
I20260926 11:40:36.026212 9898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:36.013871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:33507 (local address 127.6.96.2:33039)
0926 11:40:36.014284 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:36.014323 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:36.015570 (+ 1247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:36.016072 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:36.016079 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:36.016425 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:36.016870 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:36.016883 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:36.017792 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:36.017795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:36.018671 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:36.018681 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:36.019276 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:36.019281 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:36.019471 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:36.020089 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:36.020109 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:36.021818 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:36.023643 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:36.023649 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:36.023663 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:36.023913 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:36.024158 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:36.024161 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:36.024163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:36.024292 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:36.024645 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:36.024653 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:36.024916 (+ 263us) client_negotiation.cc:769] Sending connection context
0926 11:40:36.025078 (+ 162us) client_negotiation.cc:240] Negotiation successful
0926 11:40:36.025394 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":103,"threads_started":1}
I20260926 11:40:36.027318 9896 heartbeater.cc:347] Connected to a master server at 127.6.96.62:33507
I20260926 11:40:36.027545 9896 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:36.028030 9896 heartbeater.cc:511] Master 127.6.96.62:33507 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.3@KRBTEST.COM" created.
I20260926 11:40:36.029006 9571 ts_manager.cc:194] Registered new tserver with Master: c9b375e161684c72a35b88533fbcefb5 (127.6.96.2:40567)
I20260926 11:40:36.029560 9571 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.2@KRBTEST.COM'} at 127.6.96.2:33039
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
I20260926 11:40:36.046056 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:33507
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:37229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:36.154083 9901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:36.154382 9901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:36.154423 9901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:36.158527 9901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:36.158586 9901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:36.158654 9901 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:40:36.163971 9901 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:37229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:33507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:36.165014 9901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:36.166039 9901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 11:40:36.175585 9901 server_base.cc:1034] running on GCE node
W20260926 11:40:36.175520 9909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:36.175521 9906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:36.175757 9907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:36.176154 9901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:36.176769 9901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:36.177927 9901 hybrid_clock.cc:648] HybridClock initialized: now 1790422836177900 us; error 40 us; skew 500 ppm
Sep 26 11:40:36 dist-test-slave-xxxx krb5kdc[9526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422836, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:36.180557 9901 init.cc:377] Logged in from keytab as kudu/127.6.96.3@KRBTEST.COM (short username kudu)
I20260926 11:40:36.181447 9901 webserver.cc:460] Webserver started at http://127.6.96.3:41335/ using document root <none> and password file <none>
I20260926 11:40:36.182030 9901 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:36.182097 9901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:36.182298 9901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:36.184357 9901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "065ace0e38644a3eb8b54877dfe8dc79"
format_stamp: "Formatted at 2026-09-26 11:40:36 on dist-test-slave-xxxx"
server_key: "b9cd8c985416015f2b464da19a1b5580"
server_key_iv: "a76c9a994b4c24fc5d21833d7bd0fa34"
server_key_version: "encryptionkey@0"
I20260926 11:40:36.184911 9901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "065ace0e38644a3eb8b54877dfe8dc79"
format_stamp: "Formatted at 2026-09-26 11:40:36 on dist-test-slave-xxxx"
server_key: "b9cd8c985416015f2b464da19a1b5580"
server_key_iv: "a76c9a994b4c24fc5d21833d7bd0fa34"
server_key_version: "encryptionkey@0"
I20260926 11:40:36.188576 9901 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260926 11:40:36.191016 9916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:36.192279 9901 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:36.192389 9901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "065ace0e38644a3eb8b54877dfe8dc79"
format_stamp: "Formatted at 2026-09-26 11:40:36 on dist-test-slave-xxxx"
server_key: "b9cd8c985416015f2b464da19a1b5580"
server_key_iv: "a76c9a994b4c24fc5d21833d7bd0fa34"
server_key_version: "encryptionkey@0"
I20260926 11:40:36.192478 9901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:36.212469 9901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:36.215605 9901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:36.215785 9901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:36.216343 9901 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:36.217142 9901 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:36.217186 9901 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:36.217239 9901 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:36.217267 9901 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:36.225412 9901 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:43527
I20260926 11:40:36.225438 10029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:43527 every 8 connection(s)
I20260926 11:40:36.226274 9901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:40:36.231936 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 9901
I20260926 11:40:36.232051 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:40:36.232326 6528 external_mini_cluster.cc:1428] Setting key 93e7a6b27e3c2b75016c678bb0317faa
Sep 26 11:40:36 dist-test-slave-xxxx krb5kdc[9526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422836, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:36.239136 9610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:36.227990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:36449 (local address 127.6.96.62:33507)
0926 11:40:36.228124 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:36.228128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:36.229118 (+ 990us) server_negotiation.cc:407] Connection header received
0926 11:40:36.229968 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:36.229972 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:36.230019 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:36.230098 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:36.231325 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:36.231964 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:36.233178 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:36.233421 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:36.233525 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:36.236051 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:36.236054 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:36.236056 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:36.236080 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:36.237254 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:36.237784 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:36.237787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:36.237788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:36.237866 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:36.238136 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:36.238138 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:36.238139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:36.238318 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:36.238389 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:36.238868 (+ 479us) server_negotiation.cc:299] Negotiation successful
0926 11:40:36.238992 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 11:40:36.239916 10032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:36.228357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:33507 (local address 127.6.96.3:36449)
0926 11:40:36.228969 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:36.229009 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:36.229775 (+ 766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:36.230284 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:36.230291 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:36.230632 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:36.231175 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:36.231197 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:36.232129 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:36.232134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:36.233076 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:36.233085 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:36.233755 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:36.233758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:36.233917 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:36.234492 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:36.234509 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:36.235889 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:36.237359 (+ 1470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:36.237364 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:36.237377 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:36.237684 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:36.237936 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:36.237938 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:36.237939 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:36.238070 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:36.238390 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:36.238395 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:36.238624 (+ 229us) client_negotiation.cc:769] Sending connection context
0926 11:40:36.238782 (+ 158us) client_negotiation.cc:240] Negotiation successful
0926 11:40:36.239112 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":107,"threads_started":1}
I20260926 11:40:36.240979 10030 heartbeater.cc:347] Connected to a master server at 127.6.96.62:33507
I20260926 11:40:36.241183 10030 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:36.241611 10030 heartbeater.cc:511] Master 127.6.96.62:33507 requested a full tablet report, sending...
I20260926 11:40:36.242527 9571 ts_manager.cc:194] Registered new tserver with Master: 065ace0e38644a3eb8b54877dfe8dc79 (127.6.96.3:43527)
I20260926 11:40:36.243089 9571 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.3@KRBTEST.COM'} at 127.6.96.3:36449
I20260926 11:40:36.245985 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 11:40:36 dist-test-slave-xxxx krb5kdc[9526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422836, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 26 11:40:36 dist-test-slave-xxxx krb5kdc[9526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422836, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:36.269305 9610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:36.261513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38806 (local address 127.6.96.62:33507)
0926 11:40:36.261626 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:36.261630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:36.261765 (+ 135us) server_negotiation.cc:407] Connection header received
0926 11:40:36.261869 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:36.261872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:36.261913 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:36.261986 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:36.262595 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:36.263192 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:36.263991 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:36.264196 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:36.264279 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:36.266506 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:36.266509 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:36.266511 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:36.266535 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:36.267913 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:36.268311 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:36.268314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:36.268315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:36.268394 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:36.268606 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:36.268608 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:36.268609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:36.268820 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:36.268918 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:36.269093 (+ 175us) server_negotiation.cc:299] Negotiation successful
0926 11:40:36.269181 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260926 11:40:36.269667 9571 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38806
I20260926 11:40:36.270562 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 9633
I20260926 11:40:36.275259 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 9767
I20260926 11:40:36.279922 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 9901
I20260926 11:40:36.284577 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 9539
2026-09-26T11:40:36Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1988 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:40:36 dist-test-slave-xxxx krb5kdc[10042](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:40:36 dist-test-slave-xxxx krb5kdc[10042](info): set up 2 sockets
Sep 26 11:40:36 dist-test-slave-xxxx krb5kdc[10042](info): commencing operation
krb5kdc: starting...
W20260926 11:40:37.254614 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.950s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:40:37 dist-test-slave-xxxx krb5kdc[10042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:40:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:40:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:40:37.365777 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:39897
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:35161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:39897
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:37.471350 10055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:37.471607 10055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:37.471663 10055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:37.475617 10055 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:40:37.475674 10055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:37.475701 10055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:37.475739 10055 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:40:37.475767 10055 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:40:37.480969 10055 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:35161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:39897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:39897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:37.481943 10055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:37.482944 10055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:37.490909 10060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:37.490899 10063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:37.491010 10055 server_base.cc:1034] running on GCE node
W20260926 11:40:37.490899 10061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:37.491667 10055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:37.492686 10055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:37.493868 10055 hybrid_clock.cc:648] HybridClock initialized: now 1790422837493846 us; error 34 us; skew 500 ppm
Sep 26 11:40:37 dist-test-slave-xxxx krb5kdc[10042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422837, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:37.503062 10055 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:40:37.504011 10055 webserver.cc:460] Webserver started at http://127.6.96.62:35585/ using document root <none> and password file <none>
I20260926 11:40:37.504539 10055 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:37.504602 10055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:37.504792 10055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:37.506757 10055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "1994ecbedb9d492ab328263b9533b729"
format_stamp: "Formatted at 2026-09-26 11:40:37 on dist-test-slave-xxxx"
server_key: "74ace1383265cd163522172198fba977"
server_key_iv: "3ea2fd32937feb41c64989b2594c47d8"
server_key_version: "encryptionkey@0"
I20260926 11:40:37.507319 10055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "1994ecbedb9d492ab328263b9533b729"
format_stamp: "Formatted at 2026-09-26 11:40:37 on dist-test-slave-xxxx"
server_key: "74ace1383265cd163522172198fba977"
server_key_iv: "3ea2fd32937feb41c64989b2594c47d8"
server_key_version: "encryptionkey@0"
I20260926 11:40:37.510872 10055 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:40:37.513227 10070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:37.514444 10055 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:37.514570 10055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "1994ecbedb9d492ab328263b9533b729"
format_stamp: "Formatted at 2026-09-26 11:40:37 on dist-test-slave-xxxx"
server_key: "74ace1383265cd163522172198fba977"
server_key_iv: "3ea2fd32937feb41c64989b2594c47d8"
server_key_version: "encryptionkey@0"
I20260926 11:40:37.514667 10055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:37.531631 10055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:37.534471 10055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:37.534637 10055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:37.541847 10055 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:39897
I20260926 11:40:37.541870 10122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:39897 every 8 connection(s)
I20260926 11:40:37.542732 10055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:40:37.545364 10123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:37.550612 10123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1994ecbedb9d492ab328263b9533b729: Bootstrap starting.
I20260926 11:40:37.551273 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 10055
I20260926 11:40:37.551376 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:40:37.551622 6528 external_mini_cluster.cc:1428] Setting key 5e86cb12184fe73c1f083d0bb2d1835d
I20260926 11:40:37.553169 10123 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1994ecbedb9d492ab328263b9533b729: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:37.553855 10123 log.cc:824] T 00000000000000000000000000000000 P 1994ecbedb9d492ab328263b9533b729: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:37.555858 10123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1994ecbedb9d492ab328263b9533b729: No bootstrap required, opened a new log
Sep 26 11:40:37 dist-test-slave-xxxx krb5kdc[10042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:37.559885 10123 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1994ecbedb9d492ab328263b9533b729 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1994ecbedb9d492ab328263b9533b729" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 39897 } }
I20260926 11:40:37.560111 10123 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1994ecbedb9d492ab328263b9533b729 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:37.560175 10123 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1994ecbedb9d492ab328263b9533b729 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1994ecbedb9d492ab328263b9533b729, State: Initialized, Role: FOLLOWER
I20260926 11:40:37.560597 10123 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1994ecbedb9d492ab328263b9533b729 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1994ecbedb9d492ab328263b9533b729" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 39897 } }
I20260926 11:40:37.560727 10123 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1994ecbedb9d492ab328263b9533b729 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:40:37.560791 10123 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1994ecbedb9d492ab328263b9533b729 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:40:37.560890 10123 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1994ecbedb9d492ab328263b9533b729 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:37.561806 10123 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1994ecbedb9d492ab328263b9533b729 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1994ecbedb9d492ab328263b9533b729" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 39897 } }
I20260926 11:40:37.562109 10123 leader_election.cc:302] T 00000000000000000000000000000000 P 1994ecbedb9d492ab328263b9533b729 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1994ecbedb9d492ab328263b9533b729; no voters:
I20260926 11:40:37.562402 10123 leader_election.cc:288] T 00000000000000000000000000000000 P 1994ecbedb9d492ab328263b9533b729 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:40:37.562479 10128 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1994ecbedb9d492ab328263b9533b729 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:37.562680 10128 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1994ecbedb9d492ab328263b9533b729 [term 1 LEADER]: Becoming Leader. State: Replica: 1994ecbedb9d492ab328263b9533b729, State: Running, Role: LEADER
I20260926 11:40:37.563025 10128 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1994ecbedb9d492ab328263b9533b729 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1994ecbedb9d492ab328263b9533b729" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 39897 } }
I20260926 11:40:37.563668 10123 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1994ecbedb9d492ab328263b9533b729 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:40:37.564163 10130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1994ecbedb9d492ab328263b9533b729 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1994ecbedb9d492ab328263b9533b729. Latest consensus state: current_term: 1 leader_uuid: "1994ecbedb9d492ab328263b9533b729" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1994ecbedb9d492ab328263b9533b729" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 39897 } } }
I20260926 11:40:37.564278 10130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1994ecbedb9d492ab328263b9533b729 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:37.564594 10129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1994ecbedb9d492ab328263b9533b729 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1994ecbedb9d492ab328263b9533b729" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1994ecbedb9d492ab328263b9533b729" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 39897 } } }
I20260926 11:40:37.564692 10129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1994ecbedb9d492ab328263b9533b729 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:37.565126 10126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:37.552692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56712 (local address 127.6.96.62:39897)
0926 11:40:37.553098 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:37.553113 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:37.553146 (+ 33us) server_negotiation.cc:407] Connection header received
0926 11:40:37.553765 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:37.553782 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:37.554116 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:37.554480 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:37.555287 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:37.556266 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:37.557396 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:37.557846 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:37.557950 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:37.560544 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:37.560549 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:37.560563 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:37.560588 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:37.562118 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:37.562535 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:37.562542 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:37.562548 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:37.562687 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:37.563024 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:37.563027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:37.563029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:37.563438 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:37.563617 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:37.563905 (+ 288us) server_negotiation.cc:299] Negotiation successful
0926 11:40:37.564186 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":136,"threads_started":1}
I20260926 11:40:37.566133 10132 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:40:37.569054 10132 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:40:37.573989 10132 catalog_manager.cc:1347] Generated new cluster ID: 796675000fcf46eaaa1f1c4fb94cab6d
I20260926 11:40:37.574043 10132 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:40:37.585929 10132 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:40:37.587165 10132 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:40:37.595510 10132 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1994ecbedb9d492ab328263b9533b729: Generated new TSK 0
I20260926 11:40:37.596288 10132 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.96.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
I20260926 11:40:37.621918 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:39897
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:35161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:37.729153 10149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:37.729413 10149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:37.729455 10149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:37.733458 10149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:37.733515 10149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:37.733595 10149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:40:37.738704 10149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:35161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:39897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:37.739758 10149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:37.740727 10149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:37.749977 10154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:37.750137 10157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:37.749991 10155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:37.750141 10149 server_base.cc:1034] running on GCE node
I20260926 11:40:37.750617 10149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:37.751253 10149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:37.752408 10149 hybrid_clock.cc:648] HybridClock initialized: now 1790422837752393 us; error 32 us; skew 500 ppm
Sep 26 11:40:37 dist-test-slave-xxxx krb5kdc[10042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422837, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:37.754964 10149 init.cc:377] Logged in from keytab as kudu/127.6.96.1@KRBTEST.COM (short username kudu)
I20260926 11:40:37.755772 10149 webserver.cc:460] Webserver started at http://127.6.96.1:42429/ using document root <none> and password file <none>
I20260926 11:40:37.756279 10149 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:37.756340 10149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:37.756539 10149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:37.758476 10149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "66b53fed9066455a8b4c538472aa13ec"
format_stamp: "Formatted at 2026-09-26 11:40:37 on dist-test-slave-xxxx"
server_key: "083af31815ae2052ac57e1f8d4e89e25"
server_key_iv: "008587d18cd936527ae99a5d85b94c16"
server_key_version: "encryptionkey@0"
I20260926 11:40:37.759019 10149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "66b53fed9066455a8b4c538472aa13ec"
format_stamp: "Formatted at 2026-09-26 11:40:37 on dist-test-slave-xxxx"
server_key: "083af31815ae2052ac57e1f8d4e89e25"
server_key_iv: "008587d18cd936527ae99a5d85b94c16"
server_key_version: "encryptionkey@0"
I20260926 11:40:37.762392 10149 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260926 11:40:37.764806 10164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:37.766003 10149 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 11:40:37.766101 10149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "66b53fed9066455a8b4c538472aa13ec"
format_stamp: "Formatted at 2026-09-26 11:40:37 on dist-test-slave-xxxx"
server_key: "083af31815ae2052ac57e1f8d4e89e25"
server_key_iv: "008587d18cd936527ae99a5d85b94c16"
server_key_version: "encryptionkey@0"
I20260926 11:40:37.766191 10149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:37.779855 10149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:37.782948 10149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:37.783116 10149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:37.783684 10149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:37.784474 10149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:37.784519 10149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:37.784569 10149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:37.784595 10149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:37.793022 10149 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:46811
I20260926 11:40:37.793046 10277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:46811 every 8 connection(s)
I20260926 11:40:37.793928 10149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:40:37.797459 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 10149
I20260926 11:40:37.797562 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:40:37.797829 6528 external_mini_cluster.cc:1428] Setting key 2210d9323f840a78867dcbd2fec2b40f
Sep 26 11:40:37 dist-test-slave-xxxx krb5kdc[10042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422837, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.96.2@KRBTEST.COM; defaulting to no policy
I20260926 11:40:37.807498 10126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:37.795822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:52635 (local address 127.6.96.62:39897)
0926 11:40:37.795967 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:37.795971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:37.796794 (+ 823us) server_negotiation.cc:407] Connection header received
0926 11:40:37.798200 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:37.798203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:37.798249 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:37.798336 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:37.799739 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:37.800248 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:37.801460 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:37.801688 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:37.801776 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:37.804219 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:37.804222 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:37.804224 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:37.804248 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:37.805522 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:37.805984 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:37.805988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:37.805990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:37.806060 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:37.806385 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:37.806388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:37.806390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:37.806561 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:37.806645 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:37.807208 (+ 563us) server_negotiation.cc:299] Negotiation successful
0926 11:40:37.807337 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 11:40:37.808403 10280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:37.796184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:39897 (local address 127.6.96.1:52635)
0926 11:40:37.796632 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:37.796677 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:37.797989 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:37.798526 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:37.798533 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:37.798943 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:37.799579 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:37.799601 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:37.800357 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:37.800360 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:37.801336 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:37.801345 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:37.801832 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:37.801837 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:37.801995 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:37.802534 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:37.802551 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:37.804062 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:37.805626 (+ 1564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:37.805632 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:37.805647 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:37.805896 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:37.806138 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:37.806141 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:37.806143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:37.806302 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:37.806655 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:37.806661 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:37.806960 (+ 299us) client_negotiation.cc:769] Sending connection context
0926 11:40:37.807158 (+ 198us) client_negotiation.cc:240] Negotiation successful
0926 11:40:37.807479 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":130,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.2@KRBTEST.COM" created.
I20260926 11:40:37.809551 10278 heartbeater.cc:347] Connected to a master server at 127.6.96.62:39897
I20260926 11:40:37.809783 10278 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:37.810237 10278 heartbeater.cc:511] Master 127.6.96.62:39897 requested a full tablet report, sending...
I20260926 11:40:37.811668 10087 ts_manager.cc:194] Registered new tserver with Master: 66b53fed9066455a8b4c538472aa13ec (127.6.96.1:46811)
I20260926 11:40:37.812760 10087 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.1@KRBTEST.COM'} at 127.6.96.1:52635
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
I20260926 11:40:37.822048 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:39897
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:35161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:37.929492 10283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:37.929764 10283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:37.929807 10283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:37.933748 10283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:37.933800 10283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:37.933856 10283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:40:37.938891 10283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:35161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:39897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:37.939836 10283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:37.940768 10283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:37.949857 10291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:37.949869 10288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:37.949869 10289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:37.950121 10283 server_base.cc:1034] running on GCE node
I20260926 11:40:37.950641 10283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:37.951280 10283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:37.952445 10283 hybrid_clock.cc:648] HybridClock initialized: now 1790422837952422 us; error 38 us; skew 500 ppm
Sep 26 11:40:37 dist-test-slave-xxxx krb5kdc[10042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422837, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:37.954984 10283 init.cc:377] Logged in from keytab as kudu/127.6.96.2@KRBTEST.COM (short username kudu)
I20260926 11:40:37.955821 10283 webserver.cc:460] Webserver started at http://127.6.96.2:37137/ using document root <none> and password file <none>
I20260926 11:40:37.956336 10283 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:37.956398 10283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:37.956594 10283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:37.958520 10283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "3f6b5fce3255481298cca1e911e3d128"
format_stamp: "Formatted at 2026-09-26 11:40:37 on dist-test-slave-xxxx"
server_key: "6e2303c592ec45d1c267342a1b9abd72"
server_key_iv: "d84e72caf6c668550e936270a5ced214"
server_key_version: "encryptionkey@0"
I20260926 11:40:37.959055 10283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "3f6b5fce3255481298cca1e911e3d128"
format_stamp: "Formatted at 2026-09-26 11:40:37 on dist-test-slave-xxxx"
server_key: "6e2303c592ec45d1c267342a1b9abd72"
server_key_iv: "d84e72caf6c668550e936270a5ced214"
server_key_version: "encryptionkey@0"
I20260926 11:40:37.962447 10283 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260926 11:40:37.964710 10298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:37.965996 10283 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:37.966091 10283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "3f6b5fce3255481298cca1e911e3d128"
format_stamp: "Formatted at 2026-09-26 11:40:37 on dist-test-slave-xxxx"
server_key: "6e2303c592ec45d1c267342a1b9abd72"
server_key_iv: "d84e72caf6c668550e936270a5ced214"
server_key_version: "encryptionkey@0"
I20260926 11:40:37.966182 10283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:37.982074 10283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:37.985023 10283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:37.985191 10283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:37.985744 10283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:37.986548 10283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:37.986594 10283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:37.986645 10283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:37.986670 10283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:37.994791 10283 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:43965
I20260926 11:40:37.994930 10411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:43965 every 8 connection(s)
I20260926 11:40:37.995711 10283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:40:37.997555 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 10283
I20260926 11:40:37.997684 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:40:37.997982 6528 external_mini_cluster.cc:1428] Setting key 440929efb8c66ffbe84d1e0031b09758
Sep 26 11:40:38 dist-test-slave-xxxx krb5kdc[10042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422837, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.96.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.3@KRBTEST.COM" created.
I20260926 11:40:38.010687 10126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:37.997489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:45825 (local address 127.6.96.62:39897)
0926 11:40:37.997612 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:37.997616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:37.999258 (+ 1642us) server_negotiation.cc:407] Connection header received
0926 11:40:38.000255 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:38.000258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:38.000310 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:38.000381 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:38.001997 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:38.002518 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:38.004203 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:38.004419 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:38.004499 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:38.007204 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:38.007208 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:38.007211 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:38.007241 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:38.008545 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:38.009103 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:38.009106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:38.009107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:38.009177 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:38.009529 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:38.009533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:38.009535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:38.009754 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:38.009843 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:38.010395 (+ 552us) server_negotiation.cc:299] Negotiation successful
0926 11:40:38.010525 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 11:40:38.011570 10414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:37.998041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:39897 (local address 127.6.96.2:45825)
0926 11:40:37.998593 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:37.998638 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:38.000054 (+ 1416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:38.000573 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:38.000581 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:38.001139 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:38.001838 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:38.001859 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:38.002639 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:38.002643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:38.004086 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:38.004098 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:38.004570 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:38.004575 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:38.004777 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:38.005381 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:38.005403 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:38.007015 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:38.008659 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:38.008664 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:38.008678 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:38.009019 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:38.009248 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:38.009251 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:38.009253 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:38.009399 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:38.009870 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:38.009877 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:38.010142 (+ 265us) client_negotiation.cc:769] Sending connection context
0926 11:40:38.010310 (+ 168us) client_negotiation.cc:240] Negotiation successful
0926 11:40:38.010643 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":150,"threads_started":1}
I20260926 11:40:38.012992 10412 heartbeater.cc:347] Connected to a master server at 127.6.96.62:39897
I20260926 11:40:38.013249 10412 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:38.013732 10412 heartbeater.cc:511] Master 127.6.96.62:39897 requested a full tablet report, sending...
I20260926 11:40:38.014802 10087 ts_manager.cc:194] Registered new tserver with Master: 3f6b5fce3255481298cca1e911e3d128 (127.6.96.2:43965)
I20260926 11:40:38.015410 10087 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.2@KRBTEST.COM'} at 127.6.96.2:45825
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
I20260926 11:40:38.023661 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:39897
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:35161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:38.132284 10417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:38.132561 10417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:38.132618 10417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:38.136658 10417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:38.136726 10417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:38.136807 10417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:40:38.142237 10417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:35161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:39897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:38.143335 10417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:38.144351 10417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:38.153817 10422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:38.153987 10425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:38.154079 10417 server_base.cc:1034] running on GCE node
W20260926 11:40:38.153817 10423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:38.154528 10417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:38.155229 10417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:38.156384 10417 hybrid_clock.cc:648] HybridClock initialized: now 1790422838156365 us; error 34 us; skew 500 ppm
Sep 26 11:40:38 dist-test-slave-xxxx krb5kdc[10042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422838, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:38.159082 10417 init.cc:377] Logged in from keytab as kudu/127.6.96.3@KRBTEST.COM (short username kudu)
I20260926 11:40:38.159952 10417 webserver.cc:460] Webserver started at http://127.6.96.3:32803/ using document root <none> and password file <none>
I20260926 11:40:38.160480 10417 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:38.160539 10417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:38.160718 10417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:38.162717 10417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "42c45c3df22145b59cb157081f9c10cf"
format_stamp: "Formatted at 2026-09-26 11:40:38 on dist-test-slave-xxxx"
server_key: "25793f2161a9eaf801e1e4ad1c87b37c"
server_key_iv: "12f8e35d7163c416c0ea02c7e160c186"
server_key_version: "encryptionkey@0"
I20260926 11:40:38.163291 10417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "42c45c3df22145b59cb157081f9c10cf"
format_stamp: "Formatted at 2026-09-26 11:40:38 on dist-test-slave-xxxx"
server_key: "25793f2161a9eaf801e1e4ad1c87b37c"
server_key_iv: "12f8e35d7163c416c0ea02c7e160c186"
server_key_version: "encryptionkey@0"
I20260926 11:40:38.166944 10417 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 11:40:38.169392 10432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:38.170660 10417 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:38.170764 10417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "42c45c3df22145b59cb157081f9c10cf"
format_stamp: "Formatted at 2026-09-26 11:40:38 on dist-test-slave-xxxx"
server_key: "25793f2161a9eaf801e1e4ad1c87b37c"
server_key_iv: "12f8e35d7163c416c0ea02c7e160c186"
server_key_version: "encryptionkey@0"
I20260926 11:40:38.170900 10417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:38.192693 10417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:38.195812 10417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:38.195999 10417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:38.196568 10417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:38.197360 10417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:38.197402 10417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:38.197441 10417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:38.197470 10417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:38.205475 10417 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:39377
I20260926 11:40:38.205495 10545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:39377 every 8 connection(s)
I20260926 11:40:38.206311 10417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:40:38.209539 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 10417
I20260926 11:40:38.209656 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:40:38.209995 6528 external_mini_cluster.cc:1428] Setting key 0f53150b4b83c0d22bcbce8736ad9956
Sep 26 11:40:38 dist-test-slave-xxxx krb5kdc[10042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422838, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:38.222160 10126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:38.208234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:59901 (local address 127.6.96.62:39897)
0926 11:40:38.208409 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:38.208414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:38.209927 (+ 1513us) server_negotiation.cc:407] Connection header received
0926 11:40:38.211589 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:38.211592 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:38.211647 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:38.211725 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:38.213003 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:38.213788 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:38.215475 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:38.215854 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:38.215929 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:38.218933 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:38.218937 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:38.218940 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:38.218968 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:38.220340 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:38.220873 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:38.220875 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:38.220877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:38.220947 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:38.221201 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:38.221203 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:38.221204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:38.221362 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:38.221447 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:38.221942 (+ 495us) server_negotiation.cc:299] Negotiation successful
0926 11:40:38.222041 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 11:40:38.223194 10548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:38.208566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:39897 (local address 127.6.96.3:59901)
0926 11:40:38.209765 (+ 1199us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:38.209821 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:38.211278 (+ 1457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:38.211904 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:38.211913 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:38.212297 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:38.212846 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:38.212866 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:38.213936 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:38.213940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:38.215339 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:38.215351 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:38.216121 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:38.216124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:38.216316 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:38.216914 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:38.216934 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:38.218695 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:38.220453 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:38.220459 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:38.220476 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:38.220790 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:38.221010 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:38.221012 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:38.221014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:38.221136 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:38.221460 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:38.221465 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:38.221700 (+ 235us) client_negotiation.cc:769] Sending connection context
0926 11:40:38.221905 (+ 205us) client_negotiation.cc:240] Negotiation successful
0926 11:40:38.222239 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":895,"thread_start_us":129,"threads_started":1}
I20260926 11:40:38.224261 10546 heartbeater.cc:347] Connected to a master server at 127.6.96.62:39897
I20260926 11:40:38.224489 10546 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:38.224964 10546 heartbeater.cc:511] Master 127.6.96.62:39897 requested a full tablet report, sending...
I20260926 11:40:38.225854 10087 ts_manager.cc:194] Registered new tserver with Master: 42c45c3df22145b59cb157081f9c10cf (127.6.96.3:39377)
I20260926 11:40:38.226409 10087 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.3@KRBTEST.COM'} at 127.6.96.3:59901
I20260926 11:40:38.235515 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 11:40:38 dist-test-slave-xxxx krb5kdc[10042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 11:40:38 dist-test-slave-xxxx krb5kdc[10042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.1@KRBTEST.COM
I20260926 11:40:38.259982 10553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:38.250979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37624 (local address 127.6.96.1:46811)
0926 11:40:38.251322 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:38.251331 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:38.251355 (+ 24us) server_negotiation.cc:407] Connection header received
0926 11:40:38.251427 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:38.251432 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:38.251619 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:38.251737 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:38.252417 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:38.253223 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:38.254048 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:38.254431 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:38.254512 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:38.256791 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:38.256799 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:38.256816 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:38.256844 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:38.258388 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:38.258751 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:38.258758 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:38.258760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:38.258888 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:38.259103 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:38.259105 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:38.259106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:38.259356 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:38.259510 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:38.259651 (+ 141us) server_negotiation.cc:299] Negotiation successful
0926 11:40:38.259784 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":137,"threads_started":1}
Sep 26 11:40:38 dist-test-slave-xxxx krb5kdc[10042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:38.270429 10126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:38.262812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56728 (local address 127.6.96.62:39897)
0926 11:40:38.262986 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:38.262990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:38.263053 (+ 63us) server_negotiation.cc:407] Connection header received
0926 11:40:38.263187 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:38.263190 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:38.263235 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:38.263305 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:38.263908 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:38.264449 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:38.265266 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:38.265477 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:38.265566 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:38.267710 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:38.267713 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:38.267715 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:38.267738 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:38.268945 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:38.269294 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:38.269296 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:38.269297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:38.269373 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:38.269620 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:38.269623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:38.269624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:38.269868 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:38.269967 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:38.270182 (+ 215us) server_negotiation.cc:299] Negotiation successful
0926 11:40:38.270275 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260926 11:40:38.270819 10087 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56728
I20260926 11:40:38.271440 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 10149
I20260926 11:40:38.276746 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 10283
I20260926 11:40:38.281175 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 10417
I20260926 11:40:38.285585 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 10055
2026-09-26T11:40:38Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2000 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:40:38 dist-test-slave-xxxx krb5kdc[10558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:40:38 dist-test-slave-xxxx krb5kdc[10558](info): set up 2 sockets
Sep 26 11:40:38 dist-test-slave-xxxx krb5kdc[10558](info): commencing operation
krb5kdc: starting...
W20260926 11:40:39.234551 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.929s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:40:39 dist-test-slave-xxxx krb5kdc[10558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:40:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:40:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:40:39.346817 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:37909
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:41893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:37909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:39.453249 10571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:39.453514 10571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:39.453577 10571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:39.457623 10571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:40:39.457687 10571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:39.457713 10571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:39.457733 10571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:40:39.457754 10571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:40:39.463133 10571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:41893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:37909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:37909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:39.464151 10571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:39.465153 10571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:39.473203 10577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:39.473206 10579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:39.473196 10576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:39.473409 10571 server_base.cc:1034] running on GCE node
I20260926 11:40:39.474134 10571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:39.475265 10571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:39.476421 10571 hybrid_clock.cc:648] HybridClock initialized: now 1790422839476401 us; error 32 us; skew 500 ppm
Sep 26 11:40:39 dist-test-slave-xxxx krb5kdc[10558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422839, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:39.479112 10571 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:40:39.479806 10571 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:39.479872 10571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:39.480057 10571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:39.482095 10571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "58d8b8f27a0d450ab30a4c86bbb403cf"
format_stamp: "Formatted at 2026-09-26 11:40:39 on dist-test-slave-xxxx"
server_key: "8269e5671d6398cfe455632a9dbc020c"
server_key_iv: "b97b33542c7c1dba9ac7a598cb84d2fc"
server_key_version: "encryptionkey@0"
I20260926 11:40:39.482625 10571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "58d8b8f27a0d450ab30a4c86bbb403cf"
format_stamp: "Formatted at 2026-09-26 11:40:39 on dist-test-slave-xxxx"
server_key: "8269e5671d6398cfe455632a9dbc020c"
server_key_iv: "b97b33542c7c1dba9ac7a598cb84d2fc"
server_key_version: "encryptionkey@0"
I20260926 11:40:39.486140 10571 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 11:40:39.488591 10585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:39.489789 10571 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:39.489905 10571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "58d8b8f27a0d450ab30a4c86bbb403cf"
format_stamp: "Formatted at 2026-09-26 11:40:39 on dist-test-slave-xxxx"
server_key: "8269e5671d6398cfe455632a9dbc020c"
server_key_iv: "b97b33542c7c1dba9ac7a598cb84d2fc"
server_key_version: "encryptionkey@0"
I20260926 11:40:39.489997 10571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:39.520910 10571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:39.524086 10571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:39.524258 10571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:39.530676 10571 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:37909
I20260926 11:40:39.530954 10637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:37909 every 8 connection(s)
I20260926 11:40:39.531348 10571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:40:39.532258 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 10571
I20260926 11:40:39.532389 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:40:39.532641 6528 external_mini_cluster.cc:1428] Setting key a843cf4d3749b2e5ce7f4900b7962826
I20260926 11:40:39.534361 10638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:39.540601 10638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58d8b8f27a0d450ab30a4c86bbb403cf: Bootstrap starting.
Sep 26 11:40:39 dist-test-slave-xxxx krb5kdc[10558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:39.542817 10638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58d8b8f27a0d450ab30a4c86bbb403cf: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:39.543546 10638 log.cc:824] T 00000000000000000000000000000000 P 58d8b8f27a0d450ab30a4c86bbb403cf: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:39.545387 10638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58d8b8f27a0d450ab30a4c86bbb403cf: No bootstrap required, opened a new log
I20260926 11:40:39.546217 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:39.533881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46452 (local address 127.6.96.62:37909)
0926 11:40:39.534256 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:39.534271 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:39.534305 (+ 34us) server_negotiation.cc:407] Connection header received
0926 11:40:39.535006 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:39.535024 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:39.535358 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:39.535630 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:39.536459 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:39.537476 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:39.538629 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:39.539186 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:39.539310 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:39.541956 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:39.541961 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:39.541974 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:39.542002 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:39.543539 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:39.543936 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:39.543941 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:39.543945 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:39.544048 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:39.544353 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:39.544356 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:39.544358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:39.544773 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:39.544941 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:39.545213 (+ 272us) server_negotiation.cc:299] Negotiation successful
0926 11:40:39.545506 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":107,"threads_started":1}
I20260926 11:40:39.549175 10638 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58d8b8f27a0d450ab30a4c86bbb403cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d8b8f27a0d450ab30a4c86bbb403cf" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 37909 } }
I20260926 11:40:39.549387 10638 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58d8b8f27a0d450ab30a4c86bbb403cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:39.549437 10638 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58d8b8f27a0d450ab30a4c86bbb403cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58d8b8f27a0d450ab30a4c86bbb403cf, State: Initialized, Role: FOLLOWER
I20260926 11:40:39.549829 10638 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58d8b8f27a0d450ab30a4c86bbb403cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d8b8f27a0d450ab30a4c86bbb403cf" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 37909 } }
I20260926 11:40:39.549944 10638 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58d8b8f27a0d450ab30a4c86bbb403cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:40:39.550002 10638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58d8b8f27a0d450ab30a4c86bbb403cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:40:39.550099 10638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58d8b8f27a0d450ab30a4c86bbb403cf [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:39.550966 10638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58d8b8f27a0d450ab30a4c86bbb403cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d8b8f27a0d450ab30a4c86bbb403cf" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 37909 } }
I20260926 11:40:39.551280 10638 leader_election.cc:302] T 00000000000000000000000000000000 P 58d8b8f27a0d450ab30a4c86bbb403cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58d8b8f27a0d450ab30a4c86bbb403cf; no voters:
I20260926 11:40:39.551563 10638 leader_election.cc:288] T 00000000000000000000000000000000 P 58d8b8f27a0d450ab30a4c86bbb403cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:40:39.551689 10643 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58d8b8f27a0d450ab30a4c86bbb403cf [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:39.551949 10643 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58d8b8f27a0d450ab30a4c86bbb403cf [term 1 LEADER]: Becoming Leader. State: Replica: 58d8b8f27a0d450ab30a4c86bbb403cf, State: Running, Role: LEADER
I20260926 11:40:39.552255 10643 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58d8b8f27a0d450ab30a4c86bbb403cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d8b8f27a0d450ab30a4c86bbb403cf" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 37909 } }
I20260926 11:40:39.552706 10638 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58d8b8f27a0d450ab30a4c86bbb403cf [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:40:39.553807 10644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58d8b8f27a0d450ab30a4c86bbb403cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58d8b8f27a0d450ab30a4c86bbb403cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d8b8f27a0d450ab30a4c86bbb403cf" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 37909 } } }
I20260926 11:40:39.553937 10644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58d8b8f27a0d450ab30a4c86bbb403cf [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:39.554291 10652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:40:39.554656 10645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58d8b8f27a0d450ab30a4c86bbb403cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58d8b8f27a0d450ab30a4c86bbb403cf. Latest consensus state: current_term: 1 leader_uuid: "58d8b8f27a0d450ab30a4c86bbb403cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d8b8f27a0d450ab30a4c86bbb403cf" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 37909 } } }
I20260926 11:40:39.554762 10645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58d8b8f27a0d450ab30a4c86bbb403cf [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:39.557472 10652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:40:39.563043 10652 catalog_manager.cc:1347] Generated new cluster ID: cd02b52cd1e8477aa97fa13306220bb5
I20260926 11:40:39.563114 10652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:40:39.578631 10652 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:40:39.579598 10652 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:40:39.585541 10652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58d8b8f27a0d450ab30a4c86bbb403cf: Generated new TSK 0
I20260926 11:40:39.586177 10652 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.96.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
I20260926 11:40:39.621958 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:37909
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:41893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:39.751901 10664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:39.752187 10664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:39.752249 10664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:39.756465 10664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:39.756531 10664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:39.756592 10664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:40:39.761940 10664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:41893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:37909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:39.763033 10664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:39.764021 10664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:39.773058 10672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:39.773276 10664 server_base.cc:1034] running on GCE node
W20260926 11:40:39.773058 10670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:39.773118 10669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:39.773815 10664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:39.774451 10664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:39.775604 10664 hybrid_clock.cc:648] HybridClock initialized: now 1790422839775586 us; error 29 us; skew 500 ppm
Sep 26 11:40:39 dist-test-slave-xxxx krb5kdc[10558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422839, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:39.778264 10664 init.cc:377] Logged in from keytab as kudu/127.6.96.1@KRBTEST.COM (short username kudu)
I20260926 11:40:39.778988 10664 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:39.779052 10664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:39.779249 10664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:39.781222 10664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "60d18add1f354fcda7e80c8b2585e946"
format_stamp: "Formatted at 2026-09-26 11:40:39 on dist-test-slave-xxxx"
server_key: "5bee3e03048c815d87fb74b95c072590"
server_key_iv: "a741a35543f33aa11ae4770721b3bccd"
server_key_version: "encryptionkey@0"
I20260926 11:40:39.781760 10664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "60d18add1f354fcda7e80c8b2585e946"
format_stamp: "Formatted at 2026-09-26 11:40:39 on dist-test-slave-xxxx"
server_key: "5bee3e03048c815d87fb74b95c072590"
server_key_iv: "a741a35543f33aa11ae4770721b3bccd"
server_key_version: "encryptionkey@0"
I20260926 11:40:39.785403 10664 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260926 11:40:39.787777 10678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:39.789044 10664 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:40:39.789153 10664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "60d18add1f354fcda7e80c8b2585e946"
format_stamp: "Formatted at 2026-09-26 11:40:39 on dist-test-slave-xxxx"
server_key: "5bee3e03048c815d87fb74b95c072590"
server_key_iv: "a741a35543f33aa11ae4770721b3bccd"
server_key_version: "encryptionkey@0"
I20260926 11:40:39.789260 10664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:39.816560 10664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:39.819861 10664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:39.820037 10664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:39.820636 10664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:39.821457 10664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:39.821503 10664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:39.821545 10664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:39.821573 10664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:39.829905 10664 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:37231
I20260926 11:40:39.829917 10791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:37231 every 8 connection(s)
I20260926 11:40:39.830489 10664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:40:39.838948 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 10664
I20260926 11:40:39.839082 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:40:39.839398 6528 external_mini_cluster.cc:1428] Setting key 71c414292ea6ab77add15e93762d0fba
Sep 26 11:40:39 dist-test-slave-xxxx krb5kdc[10558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422839, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:39.843777 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:39.832305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:33377 (local address 127.6.96.62:37909)
0926 11:40:39.832454 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:39.832458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:39.833317 (+ 859us) server_negotiation.cc:407] Connection header received
0926 11:40:39.834641 (+ 1324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:39.834644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:39.834689 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:39.834762 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:39.835960 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:39.836501 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:39.837643 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:39.837911 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:39.838003 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:39.840562 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:39.840565 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:39.840567 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:39.840590 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:39.841835 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:39.842310 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:39.842313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:39.842314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:39.842396 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:39.842693 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:39.842697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:39.842698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:39.842908 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:39.842991 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:39.843486 (+ 495us) server_negotiation.cc:299] Negotiation successful
0926 11:40:39.843614 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 11:40:39.844624 10794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:39.832700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:37909 (local address 127.6.96.1:33377)
0926 11:40:39.833165 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:39.833208 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:39.834441 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:39.834963 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:39.834970 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:39.835329 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:39.835837 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:39.835850 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:39.836617 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:39.836620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:39.837517 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:39.837526 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:39.838041 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:39.838044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:39.838186 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:39.838735 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:39.838754 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:39.840407 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:39.841940 (+ 1533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:39.841946 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:39.841964 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:39.842221 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:39.842474 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:39.842477 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:39.842479 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:39.842613 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:39.842997 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:39.843004 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:39.843261 (+ 257us) client_negotiation.cc:769] Sending connection context
0926 11:40:39.843433 (+ 172us) client_negotiation.cc:240] Negotiation successful
0926 11:40:39.843766 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":121,"threads_started":1}
I20260926 11:40:39.845884 10792 heartbeater.cc:347] Connected to a master server at 127.6.96.62:37909
I20260926 11:40:39.846151 10792 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:39.846711 10792 heartbeater.cc:511] Master 127.6.96.62:37909 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.96.2@KRBTEST.COM; defaulting to no policy
I20260926 11:40:39.848505 10602 ts_manager.cc:194] Registered new tserver with Master: 60d18add1f354fcda7e80c8b2585e946 (127.6.96.1:37231)
I20260926 11:40:39.849643 10602 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.1@KRBTEST.COM'} at 127.6.96.1:33377
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
I20260926 11:40:39.865684 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:37909
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:41893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:39.971841 10797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:39.972095 10797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:39.972139 10797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:39.976105 10797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:39.976159 10797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:39.976223 10797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:40:39.981389 10797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:41893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:37909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:39.982394 10797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:39.983389 10797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 11:40:39.992508 10797 server_base.cc:1034] running on GCE node
W20260926 11:40:39.992456 10802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:39.992462 10805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:39.992458 10803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:39.993127 10797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:39.993767 10797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:39.994881 10797 hybrid_clock.cc:648] HybridClock initialized: now 1790422839994865 us; error 32 us; skew 500 ppm
Sep 26 11:40:39 dist-test-slave-xxxx krb5kdc[10558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422839, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:39.997473 10797 init.cc:377] Logged in from keytab as kudu/127.6.96.2@KRBTEST.COM (short username kudu)
I20260926 11:40:39.998160 10797 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:39.998229 10797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:39.998406 10797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:40.000427 10797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "3652e3e9f5074ab6ae90517eea8c8014"
format_stamp: "Formatted at 2026-09-26 11:40:39 on dist-test-slave-xxxx"
server_key: "0527475815c7615bcf0f5d75b07cbd19"
server_key_iv: "9489a7afda2190e5f5f8ae9810f8a980"
server_key_version: "encryptionkey@0"
I20260926 11:40:40.000973 10797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "3652e3e9f5074ab6ae90517eea8c8014"
format_stamp: "Formatted at 2026-09-26 11:40:39 on dist-test-slave-xxxx"
server_key: "0527475815c7615bcf0f5d75b07cbd19"
server_key_iv: "9489a7afda2190e5f5f8ae9810f8a980"
server_key_version: "encryptionkey@0"
I20260926 11:40:40.004513 10797 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 11:40:40.006778 10811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:40.008080 10797 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:40:40.008179 10797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "3652e3e9f5074ab6ae90517eea8c8014"
format_stamp: "Formatted at 2026-09-26 11:40:39 on dist-test-slave-xxxx"
server_key: "0527475815c7615bcf0f5d75b07cbd19"
server_key_iv: "9489a7afda2190e5f5f8ae9810f8a980"
server_key_version: "encryptionkey@0"
I20260926 11:40:40.008265 10797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:40.028501 10797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:40.031409 10797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:40.031569 10797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:40.032076 10797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:40.032835 10797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:40.032881 10797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:40.032917 10797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:40.032938 10797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:40.040982 10797 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:35563
I20260926 11:40:40.041041 10924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:35563 every 8 connection(s)
I20260926 11:40:40.041553 10797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
Sep 26 11:40:40 dist-test-slave-xxxx krb5kdc[10558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422839, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:40.051543 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 10797
I20260926 11:40:40.051635 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:40:40.051947 6528 external_mini_cluster.cc:1428] Setting key 2f0d6d723fed4b71e525775f9a569733
I20260926 11:40:40.054528 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:40.043240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:35349 (local address 127.6.96.62:37909)
0926 11:40:40.043391 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:40.043396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:40.044222 (+ 826us) server_negotiation.cc:407] Connection header received
0926 11:40:40.045595 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:40.045599 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:40.045658 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:40.045750 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:40.046862 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.047394 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.048433 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.048642 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.048712 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:40.050991 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:40.050995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:40.050996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:40.051019 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:40.052359 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:40.052930 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:40.052934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:40.052935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:40.053014 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:40.053350 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:40.053353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:40.053354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:40.053562 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:40.053657 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:40.054275 (+ 618us) server_negotiation.cc:299] Negotiation successful
0926 11:40:40.054401 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 11:40:40.055580 10927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:40.043611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:37909 (local address 127.6.96.2:35349)
0926 11:40:40.044077 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:40.044117 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:40.045379 (+ 1262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:40.045884 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:40.045890 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:40.046261 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:40.046719 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:40.046731 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.047484 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.047487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:40.048344 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:40.048352 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.048745 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.048747 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:40.048877 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:40.049394 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:40.049409 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:40.050791 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:40.052467 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:40.052474 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:40.052489 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:40.052844 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:40.053093 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:40.053096 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:40.053098 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:40.053231 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:40.053672 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:40.053678 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:40.054027 (+ 349us) client_negotiation.cc:769] Sending connection context
0926 11:40:40.054232 (+ 205us) client_negotiation.cc:240] Negotiation successful
0926 11:40:40.054617 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":111,"threads_started":1}
I20260926 11:40:40.056628 10925 heartbeater.cc:347] Connected to a master server at 127.6.96.62:37909
I20260926 11:40:40.056855 10925 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:40.057281 10925 heartbeater.cc:511] Master 127.6.96.62:37909 requested a full tablet report, sending...
I20260926 11:40:40.058223 10602 ts_manager.cc:194] Registered new tserver with Master: 3652e3e9f5074ab6ae90517eea8c8014 (127.6.96.2:35563)
I20260926 11:40:40.058760 10602 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.2@KRBTEST.COM'} at 127.6.96.2:35349
WARNING: no policy specified for kudu/127.6.96.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
I20260926 11:40:40.077178 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:37909
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:41893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:40.183868 10930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:40.184118 10930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:40.184178 10930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:40.188196 10930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:40.188259 10930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:40.188355 10930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:40:40.193609 10930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:41893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:37909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:40.194617 10930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:40.195623 10930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:40.204537 10936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:40.204583 10938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:40.204592 10935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:40.204738 10930 server_base.cc:1034] running on GCE node
I20260926 11:40:40.205231 10930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:40.205871 10930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:40.207037 10930 hybrid_clock.cc:648] HybridClock initialized: now 1790422840207017 us; error 36 us; skew 500 ppm
Sep 26 11:40:40 dist-test-slave-xxxx krb5kdc[10558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422840, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:40.209682 10930 init.cc:377] Logged in from keytab as kudu/127.6.96.3@KRBTEST.COM (short username kudu)
I20260926 11:40:40.210405 10930 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:40.210467 10930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:40.210629 10930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:40.212766 10930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "adfdb8f2991548e6bd0b30e5a2a8ac33"
format_stamp: "Formatted at 2026-09-26 11:40:40 on dist-test-slave-xxxx"
server_key: "1aa5f6e24eddc722dfef16578cd2622e"
server_key_iv: "491f7a393cf6dc2ea21a96def9ec078b"
server_key_version: "encryptionkey@0"
I20260926 11:40:40.213292 10930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "adfdb8f2991548e6bd0b30e5a2a8ac33"
format_stamp: "Formatted at 2026-09-26 11:40:40 on dist-test-slave-xxxx"
server_key: "1aa5f6e24eddc722dfef16578cd2622e"
server_key_iv: "491f7a393cf6dc2ea21a96def9ec078b"
server_key_version: "encryptionkey@0"
I20260926 11:40:40.217723 10930 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260926 11:40:40.220264 10944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:40.221506 10930 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 11:40:40.221621 10930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "adfdb8f2991548e6bd0b30e5a2a8ac33"
format_stamp: "Formatted at 2026-09-26 11:40:40 on dist-test-slave-xxxx"
server_key: "1aa5f6e24eddc722dfef16578cd2622e"
server_key_iv: "491f7a393cf6dc2ea21a96def9ec078b"
server_key_version: "encryptionkey@0"
I20260926 11:40:40.221735 10930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:40.235491 10930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:40.238405 10930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:40.238572 10930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:40.239200 10930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:40.240013 10930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:40.240058 10930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:40.240111 10930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:40.240135 10930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:40.248207 10930 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:43375
I20260926 11:40:40.248224 11057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:43375 every 8 connection(s)
I20260926 11:40:40.248776 10930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:40:40.252761 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 10930
I20260926 11:40:40.252890 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:40:40.253190 6528 external_mini_cluster.cc:1428] Setting key 308fdcc864f7ed08f5c53c7da6f84804
Sep 26 11:40:40 dist-test-slave-xxxx krb5kdc[10558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422840, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:40.262055 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:40.250442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:42193 (local address 127.6.96.62:37909)
0926 11:40:40.250557 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:40.250560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:40.251424 (+ 864us) server_negotiation.cc:407] Connection header received
0926 11:40:40.252788 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:40.252791 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:40.252848 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:40.252978 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:40.254430 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.255003 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.256161 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.256375 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.256461 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:40.258896 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:40.258899 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:40.258901 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:40.258925 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:40.260294 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:40.260746 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:40.260748 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:40.260750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:40.260818 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:40.261073 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:40.261075 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:40.261076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:40.261235 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:40.261308 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:40.261808 (+ 500us) server_negotiation.cc:299] Negotiation successful
0926 11:40:40.261925 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 11:40:40.262811 11060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:40.250783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:37909 (local address 127.6.96.3:42193)
0926 11:40:40.251274 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:40.251316 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:40.252587 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:40.253223 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:40.253233 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:40.253723 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:40.254279 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:40.254299 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.255112 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.255116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:40.256050 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:40.256060 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.256506 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.256510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:40.256677 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:40.257224 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:40.257240 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:40.258708 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:40.260399 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:40.260404 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:40.260416 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:40.260673 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:40.260880 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:40.260882 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:40.260883 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:40.261011 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:40.261303 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:40.261309 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:40.261565 (+ 256us) client_negotiation.cc:769] Sending connection context
0926 11:40:40.261721 (+ 156us) client_negotiation.cc:240] Negotiation successful
0926 11:40:40.261991 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":117,"threads_started":1}
I20260926 11:40:40.263983 11058 heartbeater.cc:347] Connected to a master server at 127.6.96.62:37909
I20260926 11:40:40.264209 11058 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:40.264663 11058 heartbeater.cc:511] Master 127.6.96.62:37909 requested a full tablet report, sending...
I20260926 11:40:40.265702 10602 ts_manager.cc:194] Registered new tserver with Master: adfdb8f2991548e6bd0b30e5a2a8ac33 (127.6.96.3:43375)
I20260926 11:40:40.266336 10602 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.3@KRBTEST.COM'} at 127.6.96.3:42193
I20260926 11:40:40.266945 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 11:40:40.275543 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:40.268732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46454 (local address 127.6.96.62:37909)
0926 11:40:40.268845 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:40.268849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:40.269002 (+ 153us) server_negotiation.cc:407] Connection header received
0926 11:40:40.269116 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:40.269119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:40.269160 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:40.269223 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:40.269913 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.270424 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.271265 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.271483 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.271565 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:40.272780 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:40.272783 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:40.272786 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:40.272812 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:40.274043 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:40.274404 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:40.274407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:40.274409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:40.274482 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:40.274706 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:40.274710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:40.274711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:40.274966 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:40.275086 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:40.275296 (+ 210us) server_negotiation.cc:299] Negotiation successful
0926 11:40:40.275397 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 11:40:40.278200 10602 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 11:40:40.281203 10602 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:40:40.295109 11069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:40.289494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:37231 (local address 127.0.0.1:54014)
0926 11:40:40.290483 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:40.290495 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:40.290574 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:40.291138 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:40.291141 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:40.291156 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:40.291264 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:40.291270 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.292599 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.292602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:40.293678 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:40.293687 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.294555 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.294559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:40.294675 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:40.294711 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 11:40:40.294809 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 11:40:40.294949 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":933,"mutex_wait_us":46,"thread_start_us":94,"threads_started":1}
I20260926 11:40:40.295113 11072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:40.289653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54014 (local address 127.6.96.1:37231)
0926 11:40:40.290035 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:40.290043 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:40.290578 (+ 535us) server_negotiation.cc:407] Connection header received
0926 11:40:40.290669 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:40.290673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:40.290819 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:40.290981 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:40.291391 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.292470 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.293782 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.294449 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.294544 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:40.294688 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:40.294865 (+ 177us) server_negotiation.cc:299] Negotiation successful
0926 11:40:40.294965 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":90,"threads_started":1}
I20260926 11:40:40.295161 11070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:40.289124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:43375 (local address 127.0.0.1:49868)
0926 11:40:40.289906 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:40.289957 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:40.290071 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:40.290843 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:40.290849 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:40.290870 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:40.291018 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:40.291025 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.292375 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.292378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:40.293704 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:40.293713 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.294752 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.294755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:40.294863 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:40.294884 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 11:40:40.294932 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 11:40:40.295010 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"mutex_wait_us":87,"thread_start_us":83,"threads_started":1}
I20260926 11:40:40.295742 11073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:40.289771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49868 (local address 127.6.96.3:43375)
0926 11:40:40.290348 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:40.290360 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:40.290401 (+ 41us) server_negotiation.cc:407] Connection header received
0926 11:40:40.290478 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:40.290484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:40.290692 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:40.290845 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:40.291140 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.292264 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.293835 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.294623 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.295326 (+ 703us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:40.295472 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:40.295519 (+ 47us) server_negotiation.cc:299] Negotiation successful
0926 11:40:40.295601 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":143,"threads_started":1}
I20260926 11:40:40.295737 11071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:40.289242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:35563 (local address 127.0.0.1:45244)
0926 11:40:40.290160 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:40.290172 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:40.290277 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:40.291012 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:40.291015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:40.291029 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:40.291113 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:40.291119 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.292103 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.292106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:40.293702 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:40.293713 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.294939 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.294943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:40.295043 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:40.295063 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 11:40:40.295122 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 11:40:40.295625 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"mutex_wait_us":59,"thread_start_us":83,"threads_started":1}
I20260926 11:40:40.296355 11074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:40.290136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45244 (local address 127.6.96.2:35563)
0926 11:40:40.290506 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:40.290514 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:40.290532 (+ 18us) server_negotiation.cc:407] Connection header received
0926 11:40:40.290604 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:40.290609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:40.290818 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:40.290986 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:40.291205 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.291988 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.293835 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.294623 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.294723 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:40.294888 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:40.296125 (+ 1237us) server_negotiation.cc:299] Negotiation successful
0926 11:40:40.296216 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":88,"threads_started":1}
I20260926 11:40:40.298156 10992 tablet_service.cc:1467] Processing CreateTablet for tablet ad6a5ebf7638412e836df48beaa6e668 (DEFAULT_TABLE table=test-table [id=676323e038da443a8227537600b60abd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:40.298156 10726 tablet_service.cc:1467] Processing CreateTablet for tablet ad6a5ebf7638412e836df48beaa6e668 (DEFAULT_TABLE table=test-table [id=676323e038da443a8227537600b60abd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:40.298518 10859 tablet_service.cc:1467] Processing CreateTablet for tablet ad6a5ebf7638412e836df48beaa6e668 (DEFAULT_TABLE table=test-table [id=676323e038da443a8227537600b60abd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:40.299280 10726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad6a5ebf7638412e836df48beaa6e668. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:40.299284 10992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad6a5ebf7638412e836df48beaa6e668. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:40.299433 10859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad6a5ebf7638412e836df48beaa6e668. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:40.305029 11075 tablet_bootstrap.cc:492] T ad6a5ebf7638412e836df48beaa6e668 P 3652e3e9f5074ab6ae90517eea8c8014: Bootstrap starting.
I20260926 11:40:40.306418 11076 tablet_bootstrap.cc:492] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946: Bootstrap starting.
I20260926 11:40:40.307360 11075 tablet_bootstrap.cc:654] T ad6a5ebf7638412e836df48beaa6e668 P 3652e3e9f5074ab6ae90517eea8c8014: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:40.307556 11077 tablet_bootstrap.cc:492] T ad6a5ebf7638412e836df48beaa6e668 P adfdb8f2991548e6bd0b30e5a2a8ac33: Bootstrap starting.
I20260926 11:40:40.308530 11076 tablet_bootstrap.cc:654] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:40.308650 11075 log.cc:824] T ad6a5ebf7638412e836df48beaa6e668 P 3652e3e9f5074ab6ae90517eea8c8014: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:40.309190 11077 tablet_bootstrap.cc:654] T ad6a5ebf7638412e836df48beaa6e668 P adfdb8f2991548e6bd0b30e5a2a8ac33: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:40.309643 11076 log.cc:824] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:40.310256 11077 log.cc:824] T ad6a5ebf7638412e836df48beaa6e668 P adfdb8f2991548e6bd0b30e5a2a8ac33: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:40.310664 11075 tablet_bootstrap.cc:492] T ad6a5ebf7638412e836df48beaa6e668 P 3652e3e9f5074ab6ae90517eea8c8014: No bootstrap required, opened a new log
I20260926 11:40:40.310871 11075 ts_tablet_manager.cc:1397] T ad6a5ebf7638412e836df48beaa6e668 P 3652e3e9f5074ab6ae90517eea8c8014: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 11:40:40.311540 11076 tablet_bootstrap.cc:492] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946: No bootstrap required, opened a new log
I20260926 11:40:40.311756 11076 ts_tablet_manager.cc:1397] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 11:40:40.312175 11077 tablet_bootstrap.cc:492] T ad6a5ebf7638412e836df48beaa6e668 P adfdb8f2991548e6bd0b30e5a2a8ac33: No bootstrap required, opened a new log
I20260926 11:40:40.312345 11077 ts_tablet_manager.cc:1397] T ad6a5ebf7638412e836df48beaa6e668 P adfdb8f2991548e6bd0b30e5a2a8ac33: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 11:40:40.314715 11075 raft_consensus.cc:348] T ad6a5ebf7638412e836df48beaa6e668 P 3652e3e9f5074ab6ae90517eea8c8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfdb8f2991548e6bd0b30e5a2a8ac33" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 43375 } } peers { permanent_uuid: "3652e3e9f5074ab6ae90517eea8c8014" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35563 } } peers { permanent_uuid: "60d18add1f354fcda7e80c8b2585e946" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 37231 } }
I20260926 11:40:40.315032 11075 raft_consensus.cc:374] T ad6a5ebf7638412e836df48beaa6e668 P 3652e3e9f5074ab6ae90517eea8c8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:40.315089 11075 raft_consensus.cc:729] T ad6a5ebf7638412e836df48beaa6e668 P 3652e3e9f5074ab6ae90517eea8c8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3652e3e9f5074ab6ae90517eea8c8014, State: Initialized, Role: FOLLOWER
I20260926 11:40:40.314936 11077 raft_consensus.cc:348] T ad6a5ebf7638412e836df48beaa6e668 P adfdb8f2991548e6bd0b30e5a2a8ac33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfdb8f2991548e6bd0b30e5a2a8ac33" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 43375 } } peers { permanent_uuid: "3652e3e9f5074ab6ae90517eea8c8014" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35563 } } peers { permanent_uuid: "60d18add1f354fcda7e80c8b2585e946" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 37231 } }
I20260926 11:40:40.315166 11077 raft_consensus.cc:374] T ad6a5ebf7638412e836df48beaa6e668 P adfdb8f2991548e6bd0b30e5a2a8ac33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:40.315220 11077 raft_consensus.cc:729] T ad6a5ebf7638412e836df48beaa6e668 P adfdb8f2991548e6bd0b30e5a2a8ac33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adfdb8f2991548e6bd0b30e5a2a8ac33, State: Initialized, Role: FOLLOWER
I20260926 11:40:40.315382 11076 raft_consensus.cc:348] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfdb8f2991548e6bd0b30e5a2a8ac33" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 43375 } } peers { permanent_uuid: "3652e3e9f5074ab6ae90517eea8c8014" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35563 } } peers { permanent_uuid: "60d18add1f354fcda7e80c8b2585e946" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 37231 } }
I20260926 11:40:40.315544 11075 consensus_queue.cc:260] T ad6a5ebf7638412e836df48beaa6e668 P 3652e3e9f5074ab6ae90517eea8c8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfdb8f2991548e6bd0b30e5a2a8ac33" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 43375 } } peers { permanent_uuid: "3652e3e9f5074ab6ae90517eea8c8014" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35563 } } peers { permanent_uuid: "60d18add1f354fcda7e80c8b2585e946" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 37231 } }
I20260926 11:40:40.315629 11076 raft_consensus.cc:374] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:40.315582 11077 consensus_queue.cc:260] T ad6a5ebf7638412e836df48beaa6e668 P adfdb8f2991548e6bd0b30e5a2a8ac33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfdb8f2991548e6bd0b30e5a2a8ac33" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 43375 } } peers { permanent_uuid: "3652e3e9f5074ab6ae90517eea8c8014" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35563 } } peers { permanent_uuid: "60d18add1f354fcda7e80c8b2585e946" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 37231 } }
I20260926 11:40:40.315677 11076 raft_consensus.cc:729] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60d18add1f354fcda7e80c8b2585e946, State: Initialized, Role: FOLLOWER
I20260926 11:40:40.316197 10925 heartbeater.cc:503] Master 127.6.96.62:37909 was elected leader, sending a full tablet report...
I20260926 11:40:40.316460 11077 ts_tablet_manager.cc:1428] T ad6a5ebf7638412e836df48beaa6e668 P adfdb8f2991548e6bd0b30e5a2a8ac33: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260926 11:40:40.316476 11075 ts_tablet_manager.cc:1428] T ad6a5ebf7638412e836df48beaa6e668 P 3652e3e9f5074ab6ae90517eea8c8014: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 11:40:40.316781 11058 heartbeater.cc:503] Master 127.6.96.62:37909 was elected leader, sending a full tablet report...
I20260926 11:40:40.316721 11076 consensus_queue.cc:260] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfdb8f2991548e6bd0b30e5a2a8ac33" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 43375 } } peers { permanent_uuid: "3652e3e9f5074ab6ae90517eea8c8014" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35563 } } peers { permanent_uuid: "60d18add1f354fcda7e80c8b2585e946" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 37231 } }
I20260926 11:40:40.317312 10792 heartbeater.cc:503] Master 127.6.96.62:37909 was elected leader, sending a full tablet report...
I20260926 11:40:40.317565 11076 ts_tablet_manager.cc:1428] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260926 11:40:40.331552 10793 tablet.cc:2369] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:40:40.369323 11083 raft_consensus.cc:482] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:40:40.369529 11083 raft_consensus.cc:504] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfdb8f2991548e6bd0b30e5a2a8ac33" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 43375 } } peers { permanent_uuid: "3652e3e9f5074ab6ae90517eea8c8014" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35563 } } peers { permanent_uuid: "60d18add1f354fcda7e80c8b2585e946" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 37231 } }
I20260926 11:40:40.370756 11083 leader_election.cc:288] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adfdb8f2991548e6bd0b30e5a2a8ac33 (127.6.96.3:43375), 3652e3e9f5074ab6ae90517eea8c8014 (127.6.96.2:35563)
I20260926 11:40:40.374687 11074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:40.371101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:52075 (local address 127.6.96.2:35563)
0926 11:40:40.371308 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:40.371312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:40.371331 (+ 19us) server_negotiation.cc:407] Connection header received
0926 11:40:40.371472 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:40.371475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:40.371524 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:40.371613 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:40.371885 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.372519 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.373719 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.374134 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.374221 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:40.374283 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:40.374507 (+ 224us) server_negotiation.cc:299] Negotiation successful
0926 11:40:40.374578 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260926 11:40:40.374701 11085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:40.371000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:43375 (local address 127.6.96.1:53573)
0926 11:40:40.371364 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:40.371377 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:40.371485 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:40.371736 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:40.371739 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:40.371751 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:40.371845 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:40.371851 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.372708 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.372711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:40.373605 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:40.373614 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.374255 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.374258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:40.374396 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:40.374414 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 11:40:40.374489 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 11:40:40.374561 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":76,"thread_start_us":94,"threads_started":1}
I20260926 11:40:40.374691 11073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:40.371098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:53573 (local address 127.6.96.3:43375)
0926 11:40:40.371234 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:40.371239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:40.371462 (+ 223us) server_negotiation.cc:407] Connection header received
0926 11:40:40.371579 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:40.371582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:40.371633 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:40.371703 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:40.371954 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.372612 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.373729 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.374147 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.374240 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:40.374297 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:40.374521 (+ 224us) server_negotiation.cc:299] Negotiation successful
0926 11:40:40.374592 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 11:40:40.374707 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:40.371000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:35563 (local address 127.6.96.1:52075)
0926 11:40:40.371266 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:40.371280 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:40.371372 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:40.371627 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:40.371630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:40.371644 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:40.371758 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:40.371764 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.372643 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.372646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:40.373596 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:40.373605 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.374249 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.374253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:40.374392 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:40.374411 (+ 19us) client_negotiation.cc:769] Sending connection context
0926 11:40:40.374468 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 11:40:40.374549 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":78,"thread_start_us":82,"threads_started":1}
I20260926 11:40:40.375531 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad6a5ebf7638412e836df48beaa6e668" candidate_uuid: "60d18add1f354fcda7e80c8b2585e946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adfdb8f2991548e6bd0b30e5a2a8ac33" is_pre_election: true
I20260926 11:40:40.375526 10879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad6a5ebf7638412e836df48beaa6e668" candidate_uuid: "60d18add1f354fcda7e80c8b2585e946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3652e3e9f5074ab6ae90517eea8c8014" is_pre_election: true
I20260926 11:40:40.375838 10879 raft_consensus.cc:2454] T ad6a5ebf7638412e836df48beaa6e668 P 3652e3e9f5074ab6ae90517eea8c8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60d18add1f354fcda7e80c8b2585e946 in term 0.
I20260926 11:40:40.375838 11012 raft_consensus.cc:2454] T ad6a5ebf7638412e836df48beaa6e668 P adfdb8f2991548e6bd0b30e5a2a8ac33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60d18add1f354fcda7e80c8b2585e946 in term 0.
I20260926 11:40:40.376305 10682 leader_election.cc:302] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3652e3e9f5074ab6ae90517eea8c8014, 60d18add1f354fcda7e80c8b2585e946; no voters:
I20260926 11:40:40.376561 11083 raft_consensus.cc:2781] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:40:40.376659 11083 raft_consensus.cc:482] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:40:40.376710 11083 raft_consensus.cc:3037] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:40.377657 11083 raft_consensus.cc:504] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfdb8f2991548e6bd0b30e5a2a8ac33" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 43375 } } peers { permanent_uuid: "3652e3e9f5074ab6ae90517eea8c8014" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35563 } } peers { permanent_uuid: "60d18add1f354fcda7e80c8b2585e946" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 37231 } }
I20260926 11:40:40.377954 11083 leader_election.cc:288] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946 [CANDIDATE]: Term 1 election: Requested vote from peers adfdb8f2991548e6bd0b30e5a2a8ac33 (127.6.96.3:43375), 3652e3e9f5074ab6ae90517eea8c8014 (127.6.96.2:35563)
I20260926 11:40:40.378278 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad6a5ebf7638412e836df48beaa6e668" candidate_uuid: "60d18add1f354fcda7e80c8b2585e946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adfdb8f2991548e6bd0b30e5a2a8ac33"
I20260926 11:40:40.378322 10879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad6a5ebf7638412e836df48beaa6e668" candidate_uuid: "60d18add1f354fcda7e80c8b2585e946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3652e3e9f5074ab6ae90517eea8c8014"
I20260926 11:40:40.378391 11012 raft_consensus.cc:3037] T ad6a5ebf7638412e836df48beaa6e668 P adfdb8f2991548e6bd0b30e5a2a8ac33 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:40.378424 10879 raft_consensus.cc:3037] T ad6a5ebf7638412e836df48beaa6e668 P 3652e3e9f5074ab6ae90517eea8c8014 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:40.379477 10879 raft_consensus.cc:2454] T ad6a5ebf7638412e836df48beaa6e668 P 3652e3e9f5074ab6ae90517eea8c8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60d18add1f354fcda7e80c8b2585e946 in term 1.
I20260926 11:40:40.379494 11012 raft_consensus.cc:2454] T ad6a5ebf7638412e836df48beaa6e668 P adfdb8f2991548e6bd0b30e5a2a8ac33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60d18add1f354fcda7e80c8b2585e946 in term 1.
I20260926 11:40:40.379765 10681 leader_election.cc:302] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60d18add1f354fcda7e80c8b2585e946, adfdb8f2991548e6bd0b30e5a2a8ac33; no voters:
I20260926 11:40:40.379918 11083 raft_consensus.cc:2781] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:40.380142 11083 raft_consensus.cc:686] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946 [term 1 LEADER]: Becoming Leader. State: Replica: 60d18add1f354fcda7e80c8b2585e946, State: Running, Role: LEADER
I20260926 11:40:40.380446 11083 consensus_queue.cc:237] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfdb8f2991548e6bd0b30e5a2a8ac33" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 43375 } } peers { permanent_uuid: "3652e3e9f5074ab6ae90517eea8c8014" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35563 } } peers { permanent_uuid: "60d18add1f354fcda7e80c8b2585e946" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 37231 } }
I20260926 11:40:40.383461 10602 catalog_manager.cc:5495] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60d18add1f354fcda7e80c8b2585e946 (127.6.96.1). New cstate: current_term: 1 leader_uuid: "60d18add1f354fcda7e80c8b2585e946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfdb8f2991548e6bd0b30e5a2a8ac33" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 43375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3652e3e9f5074ab6ae90517eea8c8014" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60d18add1f354fcda7e80c8b2585e946" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 37231 } health_report { overall_health: HEALTHY } } }
I20260926 11:40:40.398895 11072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:40.395761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54022 (local address 127.6.96.1:37231)
0926 11:40:40.395868 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:40.395871 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:40.395884 (+ 13us) server_negotiation.cc:407] Connection header received
0926 11:40:40.396015 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:40.396017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:40.396071 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:40.396157 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:40:40.396343 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.396834 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.397596 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:40.397795 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:40.397862 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:40.398157 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:40:40.398564 (+ 407us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:40:40.398650 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:40.398713 (+ 63us) server_negotiation.cc:299] Negotiation successful
0926 11:40:40.398758 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 11:40:40.404862 11012 raft_consensus.cc:1261] T ad6a5ebf7638412e836df48beaa6e668 P adfdb8f2991548e6bd0b30e5a2a8ac33 [term 1 FOLLOWER]: Refusing update from remote peer 60d18add1f354fcda7e80c8b2585e946: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:40.404862 10879 raft_consensus.cc:1261] T ad6a5ebf7638412e836df48beaa6e668 P 3652e3e9f5074ab6ae90517eea8c8014 [term 1 FOLLOWER]: Refusing update from remote peer 60d18add1f354fcda7e80c8b2585e946: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:40.405603 11086 consensus_queue.cc:1035] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3652e3e9f5074ab6ae90517eea8c8014" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:40.405802 11083 consensus_queue.cc:1035] T ad6a5ebf7638412e836df48beaa6e668 P 60d18add1f354fcda7e80c8b2585e946 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adfdb8f2991548e6bd0b30e5a2a8ac33" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 43375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:40.414045 11091 mvcc.cc:205] Tried to move back new op lower bound from 7333571954290933760 to 7333571954200637440. Current Snapshot: MvccSnapshot[applied={T|T < 7333571954290933760}]
I20260926 11:40:40.421706 10602 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:40.421901 10602 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:40.424220 10602 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 58d8b8f27a0d450ab30a4c86bbb403cf: Sending DeleteTablet for 3 replicas of tablet ad6a5ebf7638412e836df48beaa6e668
I20260926 11:40:40.424914 10992 tablet_service.cc:1514] Processing DeleteTablet for tablet ad6a5ebf7638412e836df48beaa6e668 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:40 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:49868
I20260926 11:40:40.424948 10859 tablet_service.cc:1514] Processing DeleteTablet for tablet ad6a5ebf7638412e836df48beaa6e668 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:40 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:45244
I20260926 11:40:40.425046 10726 tablet_service.cc:1514] Processing DeleteTablet for tablet ad6a5ebf7638412e836df48beaa6e668 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:40 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:54014
I20260926 11:40:40.425484 11098 tablet_replica.cc:330] T ad6a5ebf7638412e836df48beaa6e668 P adfdb8f2991548e6bd0b30e5a2a8ac33: stopping tablet replica
I20260926 11:40:40.425778 11098 raft_consensus.cc:2229] T ad6a5ebf7638412e836df48beaa6e668 P adfdb8f2991548e6bd0b30e5a2a8ac33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:40.426018 11099 tablet_replica.cc:330] T ad6a5ebf7638412e836df48beaa6e668 P 3652e3e9f5074ab6ae90517eea8c8014: stopping tablet replica
I20260926 11:40:40.426033 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 10664
I20260926 11:40:40.426280 11099 raft_consensus.cc:2229] T ad6a5ebf7638412e836df48beaa6e668 P 3652e3e9f5074ab6ae90517eea8c8014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:40.426561 11099 raft_consensus.cc:2258] T ad6a5ebf7638412e836df48beaa6e668 P 3652e3e9f5074ab6ae90517eea8c8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:40.427003 11098 raft_consensus.cc:2258] T ad6a5ebf7638412e836df48beaa6e668 P adfdb8f2991548e6bd0b30e5a2a8ac33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:40.428182 11099 ts_tablet_manager.cc:1905] T ad6a5ebf7638412e836df48beaa6e668 P 3652e3e9f5074ab6ae90517eea8c8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:40.428215 11098 ts_tablet_manager.cc:1905] T ad6a5ebf7638412e836df48beaa6e668 P adfdb8f2991548e6bd0b30e5a2a8ac33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:40.430771 11098 ts_tablet_manager.cc:1918] T ad6a5ebf7638412e836df48beaa6e668 P adfdb8f2991548e6bd0b30e5a2a8ac33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:40:40.430771 11099 ts_tablet_manager.cc:1918] T ad6a5ebf7638412e836df48beaa6e668 P 3652e3e9f5074ab6ae90517eea8c8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:40:40.430888 11098 log.cc:1196] T ad6a5ebf7638412e836df48beaa6e668 P adfdb8f2991548e6bd0b30e5a2a8ac33: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/ad6a5ebf7638412e836df48beaa6e668
I20260926 11:40:40.430888 11099 log.cc:1196] T ad6a5ebf7638412e836df48beaa6e668 P 3652e3e9f5074ab6ae90517eea8c8014: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790422817500949-6528-0/minicluster-data/ts-1/wal/wals/ad6a5ebf7638412e836df48beaa6e668
I20260926 11:40:40.431216 11099 ts_tablet_manager.cc:1939] T ad6a5ebf7638412e836df48beaa6e668 P 3652e3e9f5074ab6ae90517eea8c8014: Deleting consensus metadata
I20260926 11:40:40.431216 11098 ts_tablet_manager.cc:1939] T ad6a5ebf7638412e836df48beaa6e668 P adfdb8f2991548e6bd0b30e5a2a8ac33: Deleting consensus metadata
W20260926 11:40:40.432147 10586 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.96.1:37231 (error 108)
I20260926 11:40:40.432298 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 10797
I20260926 11:40:40.432821 10588 catalog_manager.cc:4841] TS adfdb8f2991548e6bd0b30e5a2a8ac33 (127.6.96.3:43375): tablet ad6a5ebf7638412e836df48beaa6e668 (table test-table [id=676323e038da443a8227537600b60abd]) successfully deleted
I20260926 11:40:40.433769 11071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:40.433507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:37231 (local address 127.0.0.1:54030)
0926 11:40:40.433620 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:40.433686 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.1:37231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 11:40:40.434168 10586 catalog_manager.cc:4584] TS 60d18add1f354fcda7e80c8b2585e946 (127.6.96.1:37231): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ad6a5ebf7638412e836df48beaa6e668: Network error: Client connection negotiation failed: client connection to 127.6.96.1:37231: connect: Connection refused (error 111)
I20260926 11:40:40.436893 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 10930
I20260926 11:40:40.439097 11071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:40.437020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:35563 (local address 127.0.0.1:45250)
0926 11:40:40.438982 (+ 1962us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:40.439030 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.2:35563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1886}
I20260926 11:40:40.441601 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 10571
2026-09-26T11:40:40Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2156 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-26T11:40:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:40:40Z Disabled control of system clock
I20260926 11:40:40.468789 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:44955
--webserver_interface=127.6.96.62
--webserver_port=0
--builtin_ntp_servers=127.6.96.20:40531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:44955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 11:40:40.574187 11105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:40.574450 11105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:40.574486 11105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:40.578555 11105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:40:40.578620 11105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:40.578634 11105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:40.578649 11105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:40:40.578663 11105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:40:40.583629 11105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:40531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:44955
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:44955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:40.584715 11105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:40.586097 11105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:40.594264 11113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:40.594271 11111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:40.594256 11110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:40.594585 11105 server_base.cc:1034] running on GCE node
I20260926 11:40:40.595163 11105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:40.596218 11105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:40.597374 11105 hybrid_clock.cc:648] HybridClock initialized: now 1790422840597354 us; error 33 us; skew 500 ppm
I20260926 11:40:40.599170 11105 webserver.cc:460] Webserver started at http://127.6.96.62:33023/ using document root <none> and password file <none>
I20260926 11:40:40.599689 11105 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:40.599743 11105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:40.599900 11105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:40.601809 11105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "5872079907df47ddbe97191a1d10bdbe"
format_stamp: "Formatted at 2026-09-26 11:40:40 on dist-test-slave-xxxx"
server_key: "f50b4b05d959929725d653065da48b8d"
server_key_iv: "983a6ae0bc8f37311bc84257d0793667"
server_key_version: "encryptionkey@0"
I20260926 11:40:40.602310 11105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "5872079907df47ddbe97191a1d10bdbe"
format_stamp: "Formatted at 2026-09-26 11:40:40 on dist-test-slave-xxxx"
server_key: "f50b4b05d959929725d653065da48b8d"
server_key_iv: "983a6ae0bc8f37311bc84257d0793667"
server_key_version: "encryptionkey@0"
I20260926 11:40:40.605835 11105 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 11:40:40.608151 11119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:40.609315 11105 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 11:40:40.609424 11105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "5872079907df47ddbe97191a1d10bdbe"
format_stamp: "Formatted at 2026-09-26 11:40:40 on dist-test-slave-xxxx"
server_key: "f50b4b05d959929725d653065da48b8d"
server_key_iv: "983a6ae0bc8f37311bc84257d0793667"
server_key_version: "encryptionkey@0"
I20260926 11:40:40.609519 11105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:40.613977 11105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:40.614554 11105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:40.614703 11105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:40.621068 11105 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:44955
I20260926 11:40:40.621085 11171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:44955 every 8 connection(s)
I20260926 11:40:40.621945 11105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:40:40.623944 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 11105
I20260926 11:40:40.624058 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:40:40.624296 6528 external_mini_cluster.cc:1428] Setting key df21612ff373b8bd0ffc792c778ea1a7
I20260926 11:40:40.625293 11172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:40.629586 11175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:40.625655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35630 (local address 127.6.96.62:44955)
0926 11:40:40.626236 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:40.626251 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:40.626287 (+ 36us) server_negotiation.cc:407] Connection header received
0926 11:40:40.626954 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:40.626971 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:40.627308 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:40.627586 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:40.628378 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:40.628383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:40.628394 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:40:40.628403 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:40.628463 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:40:40.628479 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:40.628583 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:40.628684 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 11:40:40.628910 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":96,"threads_started":1}
I20260926 11:40:40.631296 11172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5872079907df47ddbe97191a1d10bdbe: Bootstrap starting.
I20260926 11:40:40.633378 11172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5872079907df47ddbe97191a1d10bdbe: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:40.634043 11172 log.cc:824] T 00000000000000000000000000000000 P 5872079907df47ddbe97191a1d10bdbe: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:40.635824 11172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5872079907df47ddbe97191a1d10bdbe: No bootstrap required, opened a new log
I20260926 11:40:40.639158 11172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5872079907df47ddbe97191a1d10bdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5872079907df47ddbe97191a1d10bdbe" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44955 } }
I20260926 11:40:40.639375 11172 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5872079907df47ddbe97191a1d10bdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:40.639426 11172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5872079907df47ddbe97191a1d10bdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5872079907df47ddbe97191a1d10bdbe, State: Initialized, Role: FOLLOWER
I20260926 11:40:40.639794 11172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5872079907df47ddbe97191a1d10bdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5872079907df47ddbe97191a1d10bdbe" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44955 } }
I20260926 11:40:40.639914 11172 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5872079907df47ddbe97191a1d10bdbe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:40:40.639971 11172 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5872079907df47ddbe97191a1d10bdbe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:40:40.640057 11172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5872079907df47ddbe97191a1d10bdbe [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:40.640865 11172 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5872079907df47ddbe97191a1d10bdbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5872079907df47ddbe97191a1d10bdbe" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44955 } }
I20260926 11:40:40.641166 11172 leader_election.cc:302] T 00000000000000000000000000000000 P 5872079907df47ddbe97191a1d10bdbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5872079907df47ddbe97191a1d10bdbe; no voters:
I20260926 11:40:40.641443 11172 leader_election.cc:288] T 00000000000000000000000000000000 P 5872079907df47ddbe97191a1d10bdbe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:40:40.641522 11177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5872079907df47ddbe97191a1d10bdbe [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:40.641686 11177 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5872079907df47ddbe97191a1d10bdbe [term 1 LEADER]: Becoming Leader. State: Replica: 5872079907df47ddbe97191a1d10bdbe, State: Running, Role: LEADER
I20260926 11:40:40.641978 11177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5872079907df47ddbe97191a1d10bdbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5872079907df47ddbe97191a1d10bdbe" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44955 } }
I20260926 11:40:40.642674 11172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5872079907df47ddbe97191a1d10bdbe [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:40:40.643731 11178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5872079907df47ddbe97191a1d10bdbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5872079907df47ddbe97191a1d10bdbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5872079907df47ddbe97191a1d10bdbe" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44955 } } }
I20260926 11:40:40.643738 11179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5872079907df47ddbe97191a1d10bdbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5872079907df47ddbe97191a1d10bdbe. Latest consensus state: current_term: 1 leader_uuid: "5872079907df47ddbe97191a1d10bdbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5872079907df47ddbe97191a1d10bdbe" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44955 } } }
I20260926 11:40:40.643855 11178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5872079907df47ddbe97191a1d10bdbe [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:40.643857 11179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5872079907df47ddbe97191a1d10bdbe [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:40.644260 11186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:40:40.647125 11186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:40:40.652139 11186 catalog_manager.cc:1347] Generated new cluster ID: 70e43e4f83d44076a16703f726219389
I20260926 11:40:40.652225 11186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:40:40.675043 11186 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:40:40.675372 11186 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:40:40.683223 11186 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5872079907df47ddbe97191a1d10bdbe: Generated new TSK 0
I20260926 11:40:40.684147 11186 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:40:40.696753 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:44955
--builtin_ntp_servers=127.6.96.20:40531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 11:40:40.807344 11196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:40.807595 11196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:40.807634 11196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:40.811645 11196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:40.811691 11196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:40.811748 11196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:40:40.816468 11196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:40531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:44955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:40.817560 11196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:40.818964 11196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:40.828130 11201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:40.828181 11196 server_base.cc:1034] running on GCE node
W20260926 11:40:40.828150 11202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:40.828220 11204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:40.828712 11196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:40.829324 11196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:40.830483 11196 hybrid_clock.cc:648] HybridClock initialized: now 1790422840830464 us; error 29 us; skew 500 ppm
I20260926 11:40:40.832284 11196 webserver.cc:460] Webserver started at http://127.6.96.1:42869/ using document root <none> and password file <none>
I20260926 11:40:40.832834 11196 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:40.832897 11196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:40.833082 11196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:40.835129 11196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "8ba16af33f144b5db6e5995b64d09376"
format_stamp: "Formatted at 2026-09-26 11:40:40 on dist-test-slave-xxxx"
server_key: "79f6b422ca411e92aa3fda17ec080220"
server_key_iv: "aef502d14251eb568b97bc975825e17e"
server_key_version: "encryptionkey@0"
I20260926 11:40:40.835651 11196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "8ba16af33f144b5db6e5995b64d09376"
format_stamp: "Formatted at 2026-09-26 11:40:40 on dist-test-slave-xxxx"
server_key: "79f6b422ca411e92aa3fda17ec080220"
server_key_iv: "aef502d14251eb568b97bc975825e17e"
server_key_version: "encryptionkey@0"
I20260926 11:40:40.839340 11196 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 11:40:40.841771 11210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:40.843158 11196 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:40:40.843262 11196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "8ba16af33f144b5db6e5995b64d09376"
format_stamp: "Formatted at 2026-09-26 11:40:40 on dist-test-slave-xxxx"
server_key: "79f6b422ca411e92aa3fda17ec080220"
server_key_iv: "aef502d14251eb568b97bc975825e17e"
server_key_version: "encryptionkey@0"
I20260926 11:40:40.843365 11196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:40.847965 11196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:40.848562 11196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:40.848718 11196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:40.849241 11196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:40.850116 11196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:40.850160 11196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:40.850208 11196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:40.850236 11196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:40.858009 11196 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:39345
I20260926 11:40:40.858037 11323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:39345 every 8 connection(s)
I20260926 11:40:40.858980 11196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:40:40.862340 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 11196
I20260926 11:40:40.862454 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:40:40.862707 6528 external_mini_cluster.cc:1428] Setting key 53dc9e08e06b34b88015f03dc622280a
I20260926 11:40:40.864548 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:44955
--builtin_ntp_servers=127.6.96.20:40531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260926 11:40:40.865044 11175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:40.860679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:37361 (local address 127.6.96.62:44955)
0926 11:40:40.860816 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:40.860821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:40.861916 (+ 1095us) server_negotiation.cc:407] Connection header received
0926 11:40:40.863319 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:40.863322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:40.863364 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:40.863440 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:40.864576 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:40.864581 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:40.864583 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:40:40.864596 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:40.864620 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:40:40.864630 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:40.864719 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:40.864908 (+ 189us) server_negotiation.cc:299] Negotiation successful
0926 11:40:40.864946 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 11:40:40.866120 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:40.861119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44955 (local address 127.6.96.1:37361)
0926 11:40:40.861784 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:40.861823 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:40.863112 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:40.863626 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:40.863635 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:40.863746 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:40.864298 (+ 552us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 11:40:40.864299 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:40.864340 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 11:40:40.864410 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:40.864760 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:40.864768 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:40.864775 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 11:40:40.864939 (+ 164us) client_negotiation.cc:240] Negotiation successful
0926 11:40:40.865176 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":101,"threads_started":1}
I20260926 11:40:40.867624 11324 heartbeater.cc:347] Connected to a master server at 127.6.96.62:44955
I20260926 11:40:40.867830 11324 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:40.868227 11324 heartbeater.cc:511] Master 127.6.96.62:44955 requested a full tablet report, sending...
I20260926 11:40:40.869840 11136 ts_manager.cc:194] Registered new tserver with Master: 8ba16af33f144b5db6e5995b64d09376 (127.6.96.1:39345)
W20260926 11:40:40.971946 11327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:40.972209 11327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:40.972249 11327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:40.976344 11327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:40.976416 11327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:40.976495 11327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:40:40.981369 11327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:40531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:44955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:40.982362 11327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:40.983698 11327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:40.992694 11335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:40.992694 11332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:40.992713 11333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:40.992995 11327 server_base.cc:1034] running on GCE node
I20260926 11:40:40.993350 11327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:40.993924 11327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:40.995061 11327 hybrid_clock.cc:648] HybridClock initialized: now 1790422840995035 us; error 39 us; skew 500 ppm
I20260926 11:40:40.996731 11327 webserver.cc:460] Webserver started at http://127.6.96.2:45711/ using document root <none> and password file <none>
I20260926 11:40:40.997262 11327 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:40.997320 11327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:40.997524 11327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:40.999526 11327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "eeb10fbb3eb94e519277e422691bd3f7"
format_stamp: "Formatted at 2026-09-26 11:40:40 on dist-test-slave-xxxx"
server_key: "0967f6acb481c6e472972aa8bddd49f5"
server_key_iv: "aa43610a65f7a61137dd0163a8051cae"
server_key_version: "encryptionkey@0"
I20260926 11:40:41.000051 11327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "eeb10fbb3eb94e519277e422691bd3f7"
format_stamp: "Formatted at 2026-09-26 11:40:40 on dist-test-slave-xxxx"
server_key: "0967f6acb481c6e472972aa8bddd49f5"
server_key_iv: "aa43610a65f7a61137dd0163a8051cae"
server_key_version: "encryptionkey@0"
I20260926 11:40:41.003508 11327 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:40:41.005697 11341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:41.006803 11327 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:41.006927 11327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "eeb10fbb3eb94e519277e422691bd3f7"
format_stamp: "Formatted at 2026-09-26 11:40:40 on dist-test-slave-xxxx"
server_key: "0967f6acb481c6e472972aa8bddd49f5"
server_key_iv: "aa43610a65f7a61137dd0163a8051cae"
server_key_version: "encryptionkey@0"
I20260926 11:40:41.007014 11327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:41.011248 11327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:41.011797 11327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:41.011941 11327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:41.012419 11327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:41.013263 11327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:41.013307 11327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:41.013355 11327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:41.013381 11327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:41.021549 11327 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:46827
I20260926 11:40:41.021569 11454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:46827 every 8 connection(s)
I20260926 11:40:41.022378 11327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:40:41.028220 11175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:41.024177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:51917 (local address 127.6.96.62:44955)
0926 11:40:41.024321 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:41.024326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:41.025303 (+ 977us) server_negotiation.cc:407] Connection header received
0926 11:40:41.026537 (+ 1234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:41.026541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:41.026586 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:41.026664 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:41.027795 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:41.027799 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:41.027802 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:40:41.027815 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:41.027836 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:40:41.027844 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:41.027928 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:41.028045 (+ 117us) server_negotiation.cc:299] Negotiation successful
0926 11:40:41.028091 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 11:40:41.029044 11457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:41.024557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44955 (local address 127.6.96.2:51917)
0926 11:40:41.025160 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:41.025199 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:41.026333 (+ 1134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:41.026901 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:41.026910 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:41.027007 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:41.027537 (+ 530us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 11:40:41.027539 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:41.027580 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 11:40:41.027649 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:41.027915 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:41.027923 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:41.027928 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 11:40:41.028084 (+ 156us) client_negotiation.cc:240] Negotiation successful
0926 11:40:41.028329 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":97,"threads_started":1}
I20260926 11:40:41.030175 11455 heartbeater.cc:347] Connected to a master server at 127.6.96.62:44955
I20260926 11:40:41.030368 11455 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:41.030705 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 11327
I20260926 11:40:41.030791 11455 heartbeater.cc:511] Master 127.6.96.62:44955 requested a full tablet report, sending...
I20260926 11:40:41.030848 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:40:41.031133 6528 external_mini_cluster.cc:1428] Setting key 234ddc869eabecce58bd008297f763df
I20260926 11:40:41.031891 11136 ts_manager.cc:194] Registered new tserver with Master: eeb10fbb3eb94e519277e422691bd3f7 (127.6.96.2:46827)
I20260926 11:40:41.032961 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:44955
--builtin_ntp_servers=127.6.96.20:40531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 11:40:41.140728 11458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:41.141003 11458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:41.141055 11458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:41.145323 11458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:41.145378 11458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:41.145435 11458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:40:41.150354 11458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:40531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:44955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:41.151436 11458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:41.152733 11458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 11:40:41.161708 11458 server_base.cc:1034] running on GCE node
W20260926 11:40:41.161659 11463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:41.161738 11466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:41.161669 11464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:41.162222 11458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:41.162806 11458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:41.164017 11458 hybrid_clock.cc:648] HybridClock initialized: now 1790422841163991 us; error 40 us; skew 500 ppm
I20260926 11:40:41.165702 11458 webserver.cc:460] Webserver started at http://127.6.96.3:39595/ using document root <none> and password file <none>
I20260926 11:40:41.166242 11458 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:41.166303 11458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:41.166498 11458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:41.168488 11458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "d3739db0af874f1cbb95ce823a56c0b9"
format_stamp: "Formatted at 2026-09-26 11:40:41 on dist-test-slave-xxxx"
server_key: "b7832a4a1ca8d1674431c8b1caa90e00"
server_key_iv: "e512400cb99a9dabce15c397077bd8c9"
server_key_version: "encryptionkey@0"
I20260926 11:40:41.169001 11458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "d3739db0af874f1cbb95ce823a56c0b9"
format_stamp: "Formatted at 2026-09-26 11:40:41 on dist-test-slave-xxxx"
server_key: "b7832a4a1ca8d1674431c8b1caa90e00"
server_key_iv: "e512400cb99a9dabce15c397077bd8c9"
server_key_version: "encryptionkey@0"
I20260926 11:40:41.172490 11458 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260926 11:40:41.174655 11472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:41.175814 11458 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:41.175905 11458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "d3739db0af874f1cbb95ce823a56c0b9"
format_stamp: "Formatted at 2026-09-26 11:40:41 on dist-test-slave-xxxx"
server_key: "b7832a4a1ca8d1674431c8b1caa90e00"
server_key_iv: "e512400cb99a9dabce15c397077bd8c9"
server_key_version: "encryptionkey@0"
I20260926 11:40:41.176002 11458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:41.180370 11458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:41.180917 11458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:41.181056 11458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:41.181537 11458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:41.182351 11458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:41.182394 11458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:41.182430 11458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:41.182458 11458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:41.190435 11458 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:43313
I20260926 11:40:41.190480 11585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:43313 every 8 connection(s)
I20260926 11:40:41.191350 11458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:40:41.196578 11175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:41.192911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:59143 (local address 127.6.96.62:44955)
0926 11:40:41.193060 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:41.193064 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:41.194039 (+ 975us) server_negotiation.cc:407] Connection header received
0926 11:40:41.195262 (+ 1223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:41.195265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:41.195299 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:41.195368 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:41.196244 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:41.196247 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:41.196249 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:40:41.196259 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:41.196277 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:40:41.196285 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:41.196352 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:41.196436 (+ 84us) server_negotiation.cc:299] Negotiation successful
0926 11:40:41.196469 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 11:40:41.197355 11588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:41.193316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44955 (local address 127.6.96.3:59143)
0926 11:40:41.193905 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:41.193944 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:41.195093 (+ 1149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:41.195498 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:41.195504 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:41.195586 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:41.196028 (+ 442us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 11:40:41.196030 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:41.196067 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 11:40:41.196128 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:41.196343 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:41.196347 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:41.196351 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 11:40:41.196465 (+ 114us) client_negotiation.cc:240] Negotiation successful
0926 11:40:41.196656 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":97,"threads_started":1}
I20260926 11:40:41.198360 11586 heartbeater.cc:347] Connected to a master server at 127.6.96.62:44955
I20260926 11:40:41.198411 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 11458
I20260926 11:40:41.198506 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:40:41.198571 11586 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:41.198772 6528 external_mini_cluster.cc:1428] Setting key 9da900603682fb4d6e1be29be083242a
I20260926 11:40:41.198988 11586 heartbeater.cc:511] Master 127.6.96.62:44955 requested a full tablet report, sending...
I20260926 11:40:41.199934 11135 ts_manager.cc:194] Registered new tserver with Master: d3739db0af874f1cbb95ce823a56c0b9 (127.6.96.3:43313)
I20260926 11:40:41.200553 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 11:40:41.203651 11175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:41.202234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35640 (local address 127.6.96.62:44955)
0926 11:40:41.202346 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:41.202350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:41.202511 (+ 161us) server_negotiation.cc:407] Connection header received
0926 11:40:41.202619 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:41.202622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:41.202655 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:41.202717 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:41.203309 (+ 592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:41.203312 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:41.203315 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:40:41.203325 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:41.203344 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:40:41.203353 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:41.203419 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:41.203503 (+ 84us) server_negotiation.cc:299] Negotiation successful
0926 11:40:41.203537 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 11:40:41.206126 11136 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35640:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 11:40:41.209074 11136 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:40:41.221269 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:41.217249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:43313 (local address 127.0.0.1:41472)
0926 11:40:41.217625 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:41.217679 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:41.217775 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:41.219268 (+ 1493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:41.219275 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:41.219384 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:41.220058 (+ 674us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 11:40:41.220060 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:41.220082 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 11:40:41.220189 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:41.220956 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:41.220963 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:41.220968 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 11:40:41.221065 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 11:40:41.221132 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":934,"threads_started":1}
I20260926 11:40:41.221746 11598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:41.217328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:46827 (local address 127.0.0.1:35410)
0926 11:40:41.218393 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:41.218407 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:41.218484 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:41.220715 (+ 2231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:41.220718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:41.220732 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:41.221120 (+ 388us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 11:40:41.221122 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:41.221130 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 11:40:41.221153 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:41.221527 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:41.221530 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:41.221533 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 11:40:41.221605 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 11:40:41.221632 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1007,"thread_start_us":89,"threads_started":1}
I20260926 11:40:41.221781 11599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:41.217503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41472 (local address 127.6.96.3:43313)
0926 11:40:41.218110 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:41.218122 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:41.218167 (+ 45us) server_negotiation.cc:407] Connection header received
0926 11:40:41.218802 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:41.218810 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:41.219047 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:41.219190 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:41.220705 (+ 1515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:41.220710 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:41.220722 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:40:41.220734 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:41.220779 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:40:41.220810 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:41.220904 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:41.221557 (+ 653us) server_negotiation.cc:299] Negotiation successful
0926 11:40:41.221633 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":158,"threads_started":1}
I20260926 11:40:41.221987 11600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:41.218100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35410 (local address 127.6.96.2:46827)
0926 11:40:41.219699 (+ 1599us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:41.219710 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:41.219733 (+ 23us) server_negotiation.cc:407] Connection header received
0926 11:40:41.219784 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:41.219790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:41.219976 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:41.220108 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:41.221269 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:41.221276 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:41.221287 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:40:41.221299 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:41.221341 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:40:41.221375 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:41.221479 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:41.221633 (+ 154us) server_negotiation.cc:299] Negotiation successful
0926 11:40:41.221703 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1498,"thread_start_us":136,"threads_started":1}
I20260926 11:40:41.223291 11601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:41.219037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:39345 (local address 127.0.0.1:55618)
0926 11:40:41.219376 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:41.219389 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:41.219467 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:41.221774 (+ 2307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:41.221777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:41.221793 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:41.222677 (+ 884us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 11:40:41.222678 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:41.222686 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 11:40:41.222706 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:41.223051 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:41.223054 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:41.223057 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 11:40:41.223138 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 11:40:41.223170 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":84,"threads_started":1}
I20260926 11:40:41.223376 11602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:41.220392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55618 (local address 127.6.96.1:39345)
0926 11:40:41.220689 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:41.220698 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:41.220716 (+ 18us) server_negotiation.cc:407] Connection header received
0926 11:40:41.220767 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:41.220772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:41.221189 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:41.221313 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:41.222862 (+ 1549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:41.222868 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:41.222878 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:40:41.222890 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:41.222929 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:40:41.222961 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:41.223058 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:41.223146 (+ 88us) server_negotiation.cc:299] Negotiation successful
0926 11:40:41.223207 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":114,"threads_started":1}
I20260926 11:40:41.224612 11520 tablet_service.cc:1467] Processing CreateTablet for tablet 45a890dc30334a93acc8c6d5f6631673 (DEFAULT_TABLE table=test-table [id=5cecadd3c51b45fe967cdb2be1fe21e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:41.224663 11389 tablet_service.cc:1467] Processing CreateTablet for tablet 45a890dc30334a93acc8c6d5f6631673 (DEFAULT_TABLE table=test-table [id=5cecadd3c51b45fe967cdb2be1fe21e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:41.225569 11389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45a890dc30334a93acc8c6d5f6631673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:41.225605 11520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45a890dc30334a93acc8c6d5f6631673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:41.225802 11258 tablet_service.cc:1467] Processing CreateTablet for tablet 45a890dc30334a93acc8c6d5f6631673 (DEFAULT_TABLE table=test-table [id=5cecadd3c51b45fe967cdb2be1fe21e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:41.226691 11258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45a890dc30334a93acc8c6d5f6631673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:41.232321 11604 tablet_bootstrap.cc:492] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9: Bootstrap starting.
I20260926 11:40:41.233484 11605 tablet_bootstrap.cc:492] T 45a890dc30334a93acc8c6d5f6631673 P 8ba16af33f144b5db6e5995b64d09376: Bootstrap starting.
I20260926 11:40:41.234243 11604 tablet_bootstrap.cc:654] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:41.234992 11604 log.cc:824] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:41.235248 11603 tablet_bootstrap.cc:492] T 45a890dc30334a93acc8c6d5f6631673 P eeb10fbb3eb94e519277e422691bd3f7: Bootstrap starting.
I20260926 11:40:41.235314 11605 tablet_bootstrap.cc:654] T 45a890dc30334a93acc8c6d5f6631673 P 8ba16af33f144b5db6e5995b64d09376: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:41.236061 11605 log.cc:824] T 45a890dc30334a93acc8c6d5f6631673 P 8ba16af33f144b5db6e5995b64d09376: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:41.237041 11604 tablet_bootstrap.cc:492] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9: No bootstrap required, opened a new log
I20260926 11:40:41.237234 11604 ts_tablet_manager.cc:1397] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 11:40:41.237438 11603 tablet_bootstrap.cc:654] T 45a890dc30334a93acc8c6d5f6631673 P eeb10fbb3eb94e519277e422691bd3f7: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:41.237645 11605 tablet_bootstrap.cc:492] T 45a890dc30334a93acc8c6d5f6631673 P 8ba16af33f144b5db6e5995b64d09376: No bootstrap required, opened a new log
I20260926 11:40:41.237808 11605 ts_tablet_manager.cc:1397] T 45a890dc30334a93acc8c6d5f6631673 P 8ba16af33f144b5db6e5995b64d09376: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260926 11:40:41.238196 11603 log.cc:824] T 45a890dc30334a93acc8c6d5f6631673 P eeb10fbb3eb94e519277e422691bd3f7: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:41.240123 11603 tablet_bootstrap.cc:492] T 45a890dc30334a93acc8c6d5f6631673 P eeb10fbb3eb94e519277e422691bd3f7: No bootstrap required, opened a new log
I20260926 11:40:41.240299 11603 ts_tablet_manager.cc:1397] T 45a890dc30334a93acc8c6d5f6631673 P eeb10fbb3eb94e519277e422691bd3f7: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 11:40:41.241389 11605 raft_consensus.cc:348] T 45a890dc30334a93acc8c6d5f6631673 P 8ba16af33f144b5db6e5995b64d09376 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3739db0af874f1cbb95ce823a56c0b9" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 43313 } } peers { permanent_uuid: "eeb10fbb3eb94e519277e422691bd3f7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46827 } } peers { permanent_uuid: "8ba16af33f144b5db6e5995b64d09376" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 39345 } }
I20260926 11:40:41.241596 11605 raft_consensus.cc:374] T 45a890dc30334a93acc8c6d5f6631673 P 8ba16af33f144b5db6e5995b64d09376 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:41.241422 11604 raft_consensus.cc:348] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3739db0af874f1cbb95ce823a56c0b9" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 43313 } } peers { permanent_uuid: "eeb10fbb3eb94e519277e422691bd3f7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46827 } } peers { permanent_uuid: "8ba16af33f144b5db6e5995b64d09376" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 39345 } }
I20260926 11:40:41.241645 11605 raft_consensus.cc:729] T 45a890dc30334a93acc8c6d5f6631673 P 8ba16af33f144b5db6e5995b64d09376 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ba16af33f144b5db6e5995b64d09376, State: Initialized, Role: FOLLOWER
I20260926 11:40:41.241663 11604 raft_consensus.cc:374] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:41.241703 11604 raft_consensus.cc:729] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3739db0af874f1cbb95ce823a56c0b9, State: Initialized, Role: FOLLOWER
I20260926 11:40:41.242058 11605 consensus_queue.cc:260] T 45a890dc30334a93acc8c6d5f6631673 P 8ba16af33f144b5db6e5995b64d09376 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3739db0af874f1cbb95ce823a56c0b9" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 43313 } } peers { permanent_uuid: "eeb10fbb3eb94e519277e422691bd3f7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46827 } } peers { permanent_uuid: "8ba16af33f144b5db6e5995b64d09376" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 39345 } }
I20260926 11:40:41.242105 11604 consensus_queue.cc:260] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3739db0af874f1cbb95ce823a56c0b9" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 43313 } } peers { permanent_uuid: "eeb10fbb3eb94e519277e422691bd3f7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46827 } } peers { permanent_uuid: "8ba16af33f144b5db6e5995b64d09376" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 39345 } }
I20260926 11:40:41.242681 11324 heartbeater.cc:503] Master 127.6.96.62:44955 was elected leader, sending a full tablet report...
I20260926 11:40:41.242995 11605 ts_tablet_manager.cc:1428] T 45a890dc30334a93acc8c6d5f6631673 P 8ba16af33f144b5db6e5995b64d09376: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 11:40:41.244202 11604 ts_tablet_manager.cc:1428] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260926 11:40:41.244536 11586 heartbeater.cc:503] Master 127.6.96.62:44955 was elected leader, sending a full tablet report...
I20260926 11:40:41.244673 11603 raft_consensus.cc:348] T 45a890dc30334a93acc8c6d5f6631673 P eeb10fbb3eb94e519277e422691bd3f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3739db0af874f1cbb95ce823a56c0b9" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 43313 } } peers { permanent_uuid: "eeb10fbb3eb94e519277e422691bd3f7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46827 } } peers { permanent_uuid: "8ba16af33f144b5db6e5995b64d09376" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 39345 } }
I20260926 11:40:41.244865 11603 raft_consensus.cc:374] T 45a890dc30334a93acc8c6d5f6631673 P eeb10fbb3eb94e519277e422691bd3f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:41.244894 11603 raft_consensus.cc:729] T 45a890dc30334a93acc8c6d5f6631673 P eeb10fbb3eb94e519277e422691bd3f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eeb10fbb3eb94e519277e422691bd3f7, State: Initialized, Role: FOLLOWER
I20260926 11:40:41.245261 11603 consensus_queue.cc:260] T 45a890dc30334a93acc8c6d5f6631673 P eeb10fbb3eb94e519277e422691bd3f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3739db0af874f1cbb95ce823a56c0b9" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 43313 } } peers { permanent_uuid: "eeb10fbb3eb94e519277e422691bd3f7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46827 } } peers { permanent_uuid: "8ba16af33f144b5db6e5995b64d09376" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 39345 } }
I20260926 11:40:41.245882 11455 heartbeater.cc:503] Master 127.6.96.62:44955 was elected leader, sending a full tablet report...
I20260926 11:40:41.247473 11603 ts_tablet_manager.cc:1428] T 45a890dc30334a93acc8c6d5f6631673 P eeb10fbb3eb94e519277e422691bd3f7: Time spent starting tablet: real 0.007s user 0.001s sys 0.007s
I20260926 11:40:41.254274 11610 raft_consensus.cc:482] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:40:41.254407 11610 raft_consensus.cc:504] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3739db0af874f1cbb95ce823a56c0b9" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 43313 } } peers { permanent_uuid: "eeb10fbb3eb94e519277e422691bd3f7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46827 } } peers { permanent_uuid: "8ba16af33f144b5db6e5995b64d09376" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 39345 } }
I20260926 11:40:41.255435 11610 leader_election.cc:288] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eeb10fbb3eb94e519277e422691bd3f7 (127.6.96.2:46827), 8ba16af33f144b5db6e5995b64d09376 (127.6.96.1:39345)
I20260926 11:40:41.257005 11600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:41.255716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:47535 (local address 127.6.96.2:46827)
0926 11:40:41.255825 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:41.255828 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:41.255841 (+ 13us) server_negotiation.cc:407] Connection header received
0926 11:40:41.255948 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:41.255951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:41.255984 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:41.256055 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:41.256684 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:41.256687 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:41.256690 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:40:41.256700 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:41.256718 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:40:41.256727 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:41.256793 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:41.256857 (+ 64us) server_negotiation.cc:299] Negotiation successful
0926 11:40:41.256893 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 11:40:41.257005 11588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:41.255658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:46827 (local address 127.6.96.3:47535)
0926 11:40:41.255761 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:41.255776 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:41.255846 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:41.256088 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:41.256091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:41.256106 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:41.256553 (+ 447us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 11:40:41.256555 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:41.256566 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 11:40:41.256593 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:41.256787 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:41.256790 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:41.256794 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 11:40:41.256863 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 11:40:41.256893 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 11:40:41.257452 11602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:41.256035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:36021 (local address 127.6.96.1:39345)
0926 11:40:41.256147 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:41.256152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:41.256332 (+ 180us) server_negotiation.cc:407] Connection header received
0926 11:40:41.256409 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:41.256411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:41.256443 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:41.256526 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:41.257071 (+ 545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:41.257074 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:41.257077 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:40:41.257089 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:41.257107 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:40:41.257116 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:41.257197 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:41.257307 (+ 110us) server_negotiation.cc:299] Negotiation successful
0926 11:40:41.257346 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 11:40:41.257615 11409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45a890dc30334a93acc8c6d5f6631673" candidate_uuid: "d3739db0af874f1cbb95ce823a56c0b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eeb10fbb3eb94e519277e422691bd3f7" is_pre_election: true
I20260926 11:40:41.257766 11612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:41.255892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:39345 (local address 127.6.96.3:36021)
0926 11:40:41.256238 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:41.256252 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:41.256338 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:41.256548 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:41.256551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:41.256564 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:41.256928 (+ 364us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 11:40:41.256930 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:41.256940 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 11:40:41.256964 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:41.257197 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:41.257199 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:41.257202 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 11:40:41.257590 (+ 388us) client_negotiation.cc:240] Negotiation successful
0926 11:40:41.257626 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":111,"threads_started":1}
I20260926 11:40:41.257880 11409 raft_consensus.cc:2454] T 45a890dc30334a93acc8c6d5f6631673 P eeb10fbb3eb94e519277e422691bd3f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3739db0af874f1cbb95ce823a56c0b9 in term 0.
I20260926 11:40:41.258265 11278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45a890dc30334a93acc8c6d5f6631673" candidate_uuid: "d3739db0af874f1cbb95ce823a56c0b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ba16af33f144b5db6e5995b64d09376" is_pre_election: true
I20260926 11:40:41.258282 11476 leader_election.cc:302] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3739db0af874f1cbb95ce823a56c0b9, eeb10fbb3eb94e519277e422691bd3f7; no voters:
I20260926 11:40:41.258517 11610 raft_consensus.cc:2781] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:40:41.258520 11278 raft_consensus.cc:2454] T 45a890dc30334a93acc8c6d5f6631673 P 8ba16af33f144b5db6e5995b64d09376 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3739db0af874f1cbb95ce823a56c0b9 in term 0.
I20260926 11:40:41.258591 11610 raft_consensus.cc:482] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:40:41.258642 11610 raft_consensus.cc:3037] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:41.259642 11610 raft_consensus.cc:504] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3739db0af874f1cbb95ce823a56c0b9" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 43313 } } peers { permanent_uuid: "eeb10fbb3eb94e519277e422691bd3f7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46827 } } peers { permanent_uuid: "8ba16af33f144b5db6e5995b64d09376" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 39345 } }
I20260926 11:40:41.259922 11610 leader_election.cc:288] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9 [CANDIDATE]: Term 1 election: Requested vote from peers eeb10fbb3eb94e519277e422691bd3f7 (127.6.96.2:46827), 8ba16af33f144b5db6e5995b64d09376 (127.6.96.1:39345)
I20260926 11:40:41.260223 11409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45a890dc30334a93acc8c6d5f6631673" candidate_uuid: "d3739db0af874f1cbb95ce823a56c0b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eeb10fbb3eb94e519277e422691bd3f7"
I20260926 11:40:41.260285 11278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45a890dc30334a93acc8c6d5f6631673" candidate_uuid: "d3739db0af874f1cbb95ce823a56c0b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ba16af33f144b5db6e5995b64d09376"
I20260926 11:40:41.260334 11409 raft_consensus.cc:3037] T 45a890dc30334a93acc8c6d5f6631673 P eeb10fbb3eb94e519277e422691bd3f7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:41.260380 11278 raft_consensus.cc:3037] T 45a890dc30334a93acc8c6d5f6631673 P 8ba16af33f144b5db6e5995b64d09376 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:41.261502 11409 raft_consensus.cc:2454] T 45a890dc30334a93acc8c6d5f6631673 P eeb10fbb3eb94e519277e422691bd3f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3739db0af874f1cbb95ce823a56c0b9 in term 1.
I20260926 11:40:41.261502 11278 raft_consensus.cc:2454] T 45a890dc30334a93acc8c6d5f6631673 P 8ba16af33f144b5db6e5995b64d09376 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3739db0af874f1cbb95ce823a56c0b9 in term 1.
I20260926 11:40:41.261754 11476 leader_election.cc:302] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ba16af33f144b5db6e5995b64d09376, d3739db0af874f1cbb95ce823a56c0b9; no voters:
I20260926 11:40:41.261879 11610 raft_consensus.cc:2781] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:41.262094 11610 raft_consensus.cc:686] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9 [term 1 LEADER]: Becoming Leader. State: Replica: d3739db0af874f1cbb95ce823a56c0b9, State: Running, Role: LEADER
I20260926 11:40:41.262384 11610 consensus_queue.cc:237] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3739db0af874f1cbb95ce823a56c0b9" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 43313 } } peers { permanent_uuid: "eeb10fbb3eb94e519277e422691bd3f7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46827 } } peers { permanent_uuid: "8ba16af33f144b5db6e5995b64d09376" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 39345 } }
I20260926 11:40:41.265064 11134 catalog_manager.cc:5495] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3739db0af874f1cbb95ce823a56c0b9 (127.6.96.3). New cstate: current_term: 1 leader_uuid: "d3739db0af874f1cbb95ce823a56c0b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3739db0af874f1cbb95ce823a56c0b9" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eeb10fbb3eb94e519277e422691bd3f7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ba16af33f144b5db6e5995b64d09376" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 39345 } health_report { overall_health: UNKNOWN } } }
W20260926 11:40:41.273351 11456 tablet.cc:2369] T 45a890dc30334a93acc8c6d5f6631673 P eeb10fbb3eb94e519277e422691bd3f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:40:41.276361 11599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:41.275200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41486 (local address 127.6.96.3:43313)
0926 11:40:41.275298 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:41.275301 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:41.275313 (+ 12us) server_negotiation.cc:407] Connection header received
0926 11:40:41.275434 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:41.275436 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:41.275490 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:41.275560 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:41.276094 (+ 534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:41.276096 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:41.276098 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:40:41.276107 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:41.276124 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:40:41.276131 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:41.276196 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:41.276239 (+ 43us) server_negotiation.cc:299] Negotiation successful
0926 11:40:41.276268 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 11:40:41.281992 11278 raft_consensus.cc:1261] T 45a890dc30334a93acc8c6d5f6631673 P 8ba16af33f144b5db6e5995b64d09376 [term 1 FOLLOWER]: Refusing update from remote peer d3739db0af874f1cbb95ce823a56c0b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:41.281993 11409 raft_consensus.cc:1261] T 45a890dc30334a93acc8c6d5f6631673 P eeb10fbb3eb94e519277e422691bd3f7 [term 1 FOLLOWER]: Refusing update from remote peer d3739db0af874f1cbb95ce823a56c0b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:41.282677 11610 consensus_queue.cc:1035] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ba16af33f144b5db6e5995b64d09376" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 39345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:41.282877 11613 consensus_queue.cc:1035] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eeb10fbb3eb94e519277e422691bd3f7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:41.298646 11136 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:41.298849 11136 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:41.301216 11136 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5872079907df47ddbe97191a1d10bdbe: Sending DeleteTablet for 3 replicas of tablet 45a890dc30334a93acc8c6d5f6631673
I20260926 11:40:41.301934 11520 tablet_service.cc:1514] Processing DeleteTablet for tablet 45a890dc30334a93acc8c6d5f6631673 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:41 UTC) from {username='slave'} at 127.0.0.1:41472
I20260926 11:40:41.301982 11389 tablet_service.cc:1514] Processing DeleteTablet for tablet 45a890dc30334a93acc8c6d5f6631673 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:41 UTC) from {username='slave'} at 127.0.0.1:35410
I20260926 11:40:41.302060 11258 tablet_service.cc:1514] Processing DeleteTablet for tablet 45a890dc30334a93acc8c6d5f6631673 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:41 UTC) from {username='slave'} at 127.0.0.1:55618
I20260926 11:40:41.302395 11627 tablet_replica.cc:330] T 45a890dc30334a93acc8c6d5f6631673 P eeb10fbb3eb94e519277e422691bd3f7: stopping tablet replica
I20260926 11:40:41.302611 11626 tablet_replica.cc:330] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9: stopping tablet replica
I20260926 11:40:41.302714 11627 raft_consensus.cc:2229] T 45a890dc30334a93acc8c6d5f6631673 P eeb10fbb3eb94e519277e422691bd3f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:41.302973 11626 raft_consensus.cc:2229] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 11:40:41.303057 11627 raft_consensus.cc:2258] T 45a890dc30334a93acc8c6d5f6631673 P eeb10fbb3eb94e519277e422691bd3f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:41.303175 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 11196
I20260926 11:40:41.303457 11626 raft_consensus.cc:2258] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:41.307170 11627 ts_tablet_manager.cc:1905] T 45a890dc30334a93acc8c6d5f6631673 P eeb10fbb3eb94e519277e422691bd3f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:41.307520 11626 ts_tablet_manager.cc:1905] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:41.309346 11627 ts_tablet_manager.cc:1918] T 45a890dc30334a93acc8c6d5f6631673 P eeb10fbb3eb94e519277e422691bd3f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:40:41.309432 11627 log.cc:1196] T 45a890dc30334a93acc8c6d5f6631673 P eeb10fbb3eb94e519277e422691bd3f7: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-1/wal/wals/45a890dc30334a93acc8c6d5f6631673
I20260926 11:40:41.309748 11627 ts_tablet_manager.cc:1939] T 45a890dc30334a93acc8c6d5f6631673 P eeb10fbb3eb94e519277e422691bd3f7: Deleting consensus metadata
I20260926 11:40:41.310071 11626 ts_tablet_manager.cc:1918] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:40:41.310158 11626 log.cc:1196] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/45a890dc30334a93acc8c6d5f6631673
I20260926 11:40:41.310436 11626 ts_tablet_manager.cc:1939] T 45a890dc30334a93acc8c6d5f6631673 P d3739db0af874f1cbb95ce823a56c0b9: Deleting consensus metadata
I20260926 11:40:41.311223 11123 catalog_manager.cc:4841] TS eeb10fbb3eb94e519277e422691bd3f7 (127.6.96.2:46827): tablet 45a890dc30334a93acc8c6d5f6631673 (table test-table [id=5cecadd3c51b45fe967cdb2be1fe21e4]) successfully deleted
W20260926 11:40:41.311528 11123 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.96.1:39345 (error 108)
I20260926 11:40:41.311581 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 11327
I20260926 11:40:41.311671 11121 catalog_manager.cc:4841] TS d3739db0af874f1cbb95ce823a56c0b9 (127.6.96.3:43313): tablet 45a890dc30334a93acc8c6d5f6631673 (table test-table [id=5cecadd3c51b45fe967cdb2be1fe21e4]) successfully deleted
I20260926 11:40:41.313745 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:41.313473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:39345 (local address 127.0.0.1:55634)
0926 11:40:41.313587 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:41.313660 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.1:39345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 11:40:41.313968 11123 catalog_manager.cc:4584] TS 8ba16af33f144b5db6e5995b64d09376 (127.6.96.1:39345): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 45a890dc30334a93acc8c6d5f6631673: Network error: Client connection negotiation failed: client connection to 127.6.96.1:39345: connect: Connection refused (error 111)
I20260926 11:40:41.316740 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 11458
I20260926 11:40:41.321482 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 11105
2026-09-26T11:40:41Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (878 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260926 11:40:41.340660 6528 mini_oidc.cc:152] Starting JWKS server
I20260926 11:40:41.342163 6528 webserver.cc:460] Webserver started at https://127.0.0.1:42945/ using document root <none> and password file <none>
I20260926 11:40:41.342309 6528 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 11:40:41.342552 6528 webserver.cc:460] Webserver started at http://127.0.0.1:33275/ using document root <none> and password file <none>
2026-09-26T11:40:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:40:41Z Disabled control of system clock
I20260926 11:40:41.350402 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:46851
--webserver_interface=127.6.96.62
--webserver_port=0
--builtin_ntp_servers=127.6.96.20:33125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:46851
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42945/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790422817500949-6528-0/testchainca.cert with env {}
W20260926 11:40:41.456117 11635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:41.456398 11635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:41.456454 11635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:41.460503 11635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:40:41.460562 11635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:41.460583 11635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:41.460605 11635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:40:41.460633 11635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 11:40:41.460664 11635 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 11:40:41.460688 11635 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42945/jwks.json
I20260926 11:40:41.465955 11635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:33125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:46851
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:46851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42945/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790422817500949-6528-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:41.467067 11635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:41.468065 11635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:41.476051 11640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:41.476269 11641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:41.476315 11643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:41.476507 11635 server_base.cc:1034] running on GCE node
I20260926 11:40:41.477082 11635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:41.478175 11635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:41.479353 11635 hybrid_clock.cc:648] HybridClock initialized: now 1790422841479333 us; error 34 us; skew 500 ppm
I20260926 11:40:41.481227 11635 webserver.cc:460] Webserver started at http://127.6.96.62:39919/ using document root <none> and password file <none>
I20260926 11:40:41.481788 11635 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:41.481853 11635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:41.482043 11635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:41.484054 11635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "dd9936459f2143c9acad7151a4d6ae41"
format_stamp: "Formatted at 2026-09-26 11:40:41 on dist-test-slave-xxxx"
server_key: "ba890c484cff516f86edb45867243519"
server_key_iv: "e13a9cb5f6b2ff60d85b7df591dc4f42"
server_key_version: "encryptionkey@0"
I20260926 11:40:41.484601 11635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "dd9936459f2143c9acad7151a4d6ae41"
format_stamp: "Formatted at 2026-09-26 11:40:41 on dist-test-slave-xxxx"
server_key: "ba890c484cff516f86edb45867243519"
server_key_iv: "e13a9cb5f6b2ff60d85b7df591dc4f42"
server_key_version: "encryptionkey@0"
I20260926 11:40:41.488180 11635 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 11:40:41.490587 11649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:41.491825 11635 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:40:41.491936 11635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "dd9936459f2143c9acad7151a4d6ae41"
format_stamp: "Formatted at 2026-09-26 11:40:41 on dist-test-slave-xxxx"
server_key: "ba890c484cff516f86edb45867243519"
server_key_iv: "e13a9cb5f6b2ff60d85b7df591dc4f42"
server_key_version: "encryptionkey@0"
I20260926 11:40:41.492144 11635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:41.508662 11635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:41.509353 11635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:41.509519 11635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:41.515744 11635 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:46851
I20260926 11:40:41.515758 11701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:46851 every 8 connection(s)
I20260926 11:40:41.516625 11635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:40:41.519440 11702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:41.524722 11702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd9936459f2143c9acad7151a4d6ae41: Bootstrap starting.
I20260926 11:40:41.525300 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 11635
I20260926 11:40:41.525425 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:40:41.525666 6528 external_mini_cluster.cc:1428] Setting key 90a3266266d57b45acc79e724d0e1f33
I20260926 11:40:41.527271 11702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd9936459f2143c9acad7151a4d6ae41: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:41.527971 11702 log.cc:824] T 00000000000000000000000000000000 P dd9936459f2143c9acad7151a4d6ae41: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:41.529855 11702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd9936459f2143c9acad7151a4d6ae41: No bootstrap required, opened a new log
I20260926 11:40:41.533828 11702 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd9936459f2143c9acad7151a4d6ae41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9936459f2143c9acad7151a4d6ae41" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46851 } }
I20260926 11:40:41.534053 11702 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd9936459f2143c9acad7151a4d6ae41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:41.534117 11702 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd9936459f2143c9acad7151a4d6ae41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd9936459f2143c9acad7151a4d6ae41, State: Initialized, Role: FOLLOWER
I20260926 11:40:41.534430 11705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:41.526893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59410 (local address 127.6.96.62:46851)
0926 11:40:41.527486 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:41.527499 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:41.527542 (+ 43us) server_negotiation.cc:407] Connection header received
0926 11:40:41.528161 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:41.528179 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:41.528563 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:41.528846 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:41.529166 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:41.530144 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:41.531335 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:41.531797 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:41.531911 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:41.532710 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:41.532715 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:41.532728 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:40:41.533103 (+ 375us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:41.533157 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:40:41.533177 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:41.533250 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:41.533381 (+ 131us) server_negotiation.cc:299] Negotiation successful
0926 11:40:41.533643 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":110,"threads_started":1}
I20260926 11:40:41.534543 11702 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd9936459f2143c9acad7151a4d6ae41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9936459f2143c9acad7151a4d6ae41" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46851 } }
I20260926 11:40:41.534698 11702 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd9936459f2143c9acad7151a4d6ae41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:40:41.534754 11702 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd9936459f2143c9acad7151a4d6ae41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:40:41.534860 11702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd9936459f2143c9acad7151a4d6ae41 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:41.535866 11702 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd9936459f2143c9acad7151a4d6ae41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9936459f2143c9acad7151a4d6ae41" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46851 } }
I20260926 11:40:41.536188 11702 leader_election.cc:302] T 00000000000000000000000000000000 P dd9936459f2143c9acad7151a4d6ae41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd9936459f2143c9acad7151a4d6ae41; no voters:
I20260926 11:40:41.536535 11702 leader_election.cc:288] T 00000000000000000000000000000000 P dd9936459f2143c9acad7151a4d6ae41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:40:41.536626 11707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd9936459f2143c9acad7151a4d6ae41 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:41.536827 11707 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd9936459f2143c9acad7151a4d6ae41 [term 1 LEADER]: Becoming Leader. State: Replica: dd9936459f2143c9acad7151a4d6ae41, State: Running, Role: LEADER
I20260926 11:40:41.537129 11707 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd9936459f2143c9acad7151a4d6ae41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9936459f2143c9acad7151a4d6ae41" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46851 } }
I20260926 11:40:41.537324 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 11:40:41.537763 11702 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd9936459f2143c9acad7151a4d6ae41 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:40:41.538727 11708 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9936459f2143c9acad7151a4d6ae41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd9936459f2143c9acad7151a4d6ae41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9936459f2143c9acad7151a4d6ae41" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46851 } } }
I20260926 11:40:41.538863 11708 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9936459f2143c9acad7151a4d6ae41 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:41.539207 11716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:40:41.539165 11709 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9936459f2143c9acad7151a4d6ae41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd9936459f2143c9acad7151a4d6ae41. Latest consensus state: current_term: 1 leader_uuid: "dd9936459f2143c9acad7151a4d6ae41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9936459f2143c9acad7151a4d6ae41" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46851 } } }
I20260926 11:40:41.539963 11709 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9936459f2143c9acad7151a4d6ae41 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:41.542207 11716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:40:41.547588 11716 catalog_manager.cc:1347] Generated new cluster ID: 8d1db2169380456cae691c53a8da9c88
I20260926 11:40:41.547647 11716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:40:41.558410 11716 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:40:41.559650 11716 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:40:41.568702 11716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd9936459f2143c9acad7151a4d6ae41: Generated new TSK 0
I20260926 11:40:41.569527 11716 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 11:40:41.598146 11732 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260926 11:40:41.598274 11732 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 11:40:41.598770 11705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:41.582091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59412 (local address 127.6.96.62:46851)
0926 11:40:41.582240 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:41.582244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:41.582395 (+ 151us) server_negotiation.cc:407] Connection header received
0926 11:40:41.582583 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:41.582585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:41.582633 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:41.582717 (+ 84us) server_negotiation.cc:226] Negotiated authn=JWT
0926 11:40:41.583026 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:41.583587 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:41.584798 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:41.585033 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:41.585110 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:41.585482 (+ 372us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 11:40:41.598414 (+ 12932us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42945/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 11:40:41.598660 (+ 246us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59412: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42945/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":74}
W20260926 11:40:41.598878 11705 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59412: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42945/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260926 11:40:41.600250 11731 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:41.581965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46851 (local address 127.0.0.1:59412)
0926 11:40:41.582284 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:41.582299 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:41.582429 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:41.582741 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:41.582744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:41.582751 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0926 11:40:41.582896 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:41.582902 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:41.583681 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:41.583684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:41.584681 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:41.584691 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:41.585115 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:41.585133 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:41.585225 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:41.585278 (+ 53us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 11:40:41.598653 (+ 13375us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42945/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 11:40:41.598967 (+ 314us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.96.62:46851: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42945/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":104,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.6.96.62:46851: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42945/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260926 11:40:41.600921 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 11635
2026-09-26T11:40:41Z chronyd exiting
I20260926 11:40:41.637422 6528 test_util.cc:175] -----------------------------------------------
I20260926 11:40:41.637507 6528 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790422817500949-6528-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (298 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260926 11:40:41.638499 6528 mini_oidc.cc:152] Starting JWKS server
I20260926 11:40:41.639381 6528 webserver.cc:460] Webserver started at https://127.0.0.1:38133/ using document root <none> and password file <none>
I20260926 11:40:41.639468 6528 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 11:40:41.639674 6528 webserver.cc:460] Webserver started at http://127.0.0.1:44889/ using document root <none> and password file <none>
2026-09-26T11:40:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:40:41Z Disabled control of system clock
I20260926 11:40:41.648428 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:45851
--webserver_interface=127.6.96.62
--webserver_port=0
--builtin_ntp_servers=127.6.96.20:37743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:45851
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38133/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790422817500949-6528-0/testchainca.cert with env {}
W20260926 11:40:41.755097 11738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:41.755357 11738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:41.755395 11738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:41.759404 11738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:40:41.759459 11738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:41.759471 11738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:41.759486 11738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:40:41.759500 11738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 11:40:41.759514 11738 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 11:40:41.759526 11738 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38133/jwks.json
I20260926 11:40:41.764801 11738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:37743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:45851
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:45851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38133/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790422817500949-6528-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:41.765763 11738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:41.766685 11738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:41.774684 11746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:41.774704 11743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:41.774683 11744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:41.775045 11738 server_base.cc:1034] running on GCE node
I20260926 11:40:41.775665 11738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:41.776772 11738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:41.777953 11738 hybrid_clock.cc:648] HybridClock initialized: now 1790422841777935 us; error 37 us; skew 500 ppm
I20260926 11:40:41.779865 11738 webserver.cc:460] Webserver started at http://127.6.96.62:32943/ using document root <none> and password file <none>
I20260926 11:40:41.780432 11738 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:41.780493 11738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:41.780684 11738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:41.782665 11738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "306e78f50cc74215a791c57d9f0ba69f"
format_stamp: "Formatted at 2026-09-26 11:40:41 on dist-test-slave-xxxx"
server_key: "5b72d5a965a6d13af53979cf84565da9"
server_key_iv: "cc742fb7624a0f88ff4343e96c6fa849"
server_key_version: "encryptionkey@0"
I20260926 11:40:41.783285 11738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "306e78f50cc74215a791c57d9f0ba69f"
format_stamp: "Formatted at 2026-09-26 11:40:41 on dist-test-slave-xxxx"
server_key: "5b72d5a965a6d13af53979cf84565da9"
server_key_iv: "cc742fb7624a0f88ff4343e96c6fa849"
server_key_version: "encryptionkey@0"
I20260926 11:40:41.786788 11738 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:40:41.789139 11752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:41.790335 11738 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:41.790439 11738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "306e78f50cc74215a791c57d9f0ba69f"
format_stamp: "Formatted at 2026-09-26 11:40:41 on dist-test-slave-xxxx"
server_key: "5b72d5a965a6d13af53979cf84565da9"
server_key_iv: "cc742fb7624a0f88ff4343e96c6fa849"
server_key_version: "encryptionkey@0"
I20260926 11:40:41.790545 11738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:41.816030 11738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:41.816731 11738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:41.816903 11738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:41.823283 11738 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:45851
I20260926 11:40:41.823292 11804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:45851 every 8 connection(s)
I20260926 11:40:41.824214 11738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:40:41.827284 11805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:41.832960 11805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 306e78f50cc74215a791c57d9f0ba69f: Bootstrap starting.
I20260926 11:40:41.834065 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 11738
I20260926 11:40:41.834197 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:40:41.834476 6528 external_mini_cluster.cc:1428] Setting key 7158ff834f8cfb10df1353e5ae7c7783
I20260926 11:40:41.835902 11805 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 306e78f50cc74215a791c57d9f0ba69f: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:41.836673 11805 log.cc:824] T 00000000000000000000000000000000 P 306e78f50cc74215a791c57d9f0ba69f: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:41.838657 11805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 306e78f50cc74215a791c57d9f0ba69f: No bootstrap required, opened a new log
I20260926 11:40:41.842195 11805 raft_consensus.cc:348] T 00000000000000000000000000000000 P 306e78f50cc74215a791c57d9f0ba69f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "306e78f50cc74215a791c57d9f0ba69f" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45851 } }
I20260926 11:40:41.842418 11805 raft_consensus.cc:374] T 00000000000000000000000000000000 P 306e78f50cc74215a791c57d9f0ba69f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:41.842471 11805 raft_consensus.cc:729] T 00000000000000000000000000000000 P 306e78f50cc74215a791c57d9f0ba69f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 306e78f50cc74215a791c57d9f0ba69f, State: Initialized, Role: FOLLOWER
I20260926 11:40:41.842957 11805 consensus_queue.cc:260] T 00000000000000000000000000000000 P 306e78f50cc74215a791c57d9f0ba69f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "306e78f50cc74215a791c57d9f0ba69f" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45851 } }
I20260926 11:40:41.843101 11805 raft_consensus.cc:388] T 00000000000000000000000000000000 P 306e78f50cc74215a791c57d9f0ba69f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:40:41.843168 11805 raft_consensus.cc:482] T 00000000000000000000000000000000 P 306e78f50cc74215a791c57d9f0ba69f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:40:41.843266 11805 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 306e78f50cc74215a791c57d9f0ba69f [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:41.843565 11808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:41.835581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52686 (local address 127.6.96.62:45851)
0926 11:40:41.836251 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:41.836266 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:41.836303 (+ 37us) server_negotiation.cc:407] Connection header received
0926 11:40:41.837016 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:41.837043 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:41.837473 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:41.837821 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:41.838236 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:41.839276 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:41.840229 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:41.840763 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:41.840907 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:41.841733 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:41.841739 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:41.841752 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:40:41.842141 (+ 389us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:41.842197 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:40:41.842216 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:41.842299 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:41.842435 (+ 136us) server_negotiation.cc:299] Negotiation successful
0926 11:40:41.842692 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":127,"threads_started":1}
I20260926 11:40:41.844327 11805 raft_consensus.cc:504] T 00000000000000000000000000000000 P 306e78f50cc74215a791c57d9f0ba69f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "306e78f50cc74215a791c57d9f0ba69f" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45851 } }
I20260926 11:40:41.844683 11805 leader_election.cc:302] T 00000000000000000000000000000000 P 306e78f50cc74215a791c57d9f0ba69f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 306e78f50cc74215a791c57d9f0ba69f; no voters:
I20260926 11:40:41.844975 11805 leader_election.cc:288] T 00000000000000000000000000000000 P 306e78f50cc74215a791c57d9f0ba69f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:40:41.845075 11810 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 306e78f50cc74215a791c57d9f0ba69f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:41.845254 11810 raft_consensus.cc:686] T 00000000000000000000000000000000 P 306e78f50cc74215a791c57d9f0ba69f [term 1 LEADER]: Becoming Leader. State: Replica: 306e78f50cc74215a791c57d9f0ba69f, State: Running, Role: LEADER
I20260926 11:40:41.845552 11810 consensus_queue.cc:237] T 00000000000000000000000000000000 P 306e78f50cc74215a791c57d9f0ba69f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "306e78f50cc74215a791c57d9f0ba69f" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45851 } }
I20260926 11:40:41.846247 11805 sys_catalog.cc:564] T 00000000000000000000000000000000 P 306e78f50cc74215a791c57d9f0ba69f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:40:41.846540 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 11:40:41.846698 11810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 306e78f50cc74215a791c57d9f0ba69f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 306e78f50cc74215a791c57d9f0ba69f. Latest consensus state: current_term: 1 leader_uuid: "306e78f50cc74215a791c57d9f0ba69f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "306e78f50cc74215a791c57d9f0ba69f" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45851 } } }
I20260926 11:40:41.846802 11810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 306e78f50cc74215a791c57d9f0ba69f [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:41.847079 11811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 306e78f50cc74215a791c57d9f0ba69f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "306e78f50cc74215a791c57d9f0ba69f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "306e78f50cc74215a791c57d9f0ba69f" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45851 } } }
I20260926 11:40:41.847169 11811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 306e78f50cc74215a791c57d9f0ba69f [sys.catalog]: This master's current role is: LEADER
W20260926 11:40:41.850701 11825 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 306e78f50cc74215a791c57d9f0ba69f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 11:40:41.850765 11825 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 11:40:41.850858 11814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:40:41.851531 11814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:40:41.856369 11814 catalog_manager.cc:1347] Generated new cluster ID: 50fda186abb44004a0053d93f70520ab
I20260926 11:40:41.856436 11814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:40:41.867782 11814 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:40:41.868721 11814 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:40:41.879035 11814 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 306e78f50cc74215a791c57d9f0ba69f: Generated new TSK 0
I20260926 11:40:41.879737 11814 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 11:40:41.909412 11835 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260926 11:40:41.909505 11835 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 11:40:41.910140 11808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:41.894453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52692 (local address 127.6.96.62:45851)
0926 11:40:41.894579 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:41.894583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:41.894801 (+ 218us) server_negotiation.cc:407] Connection header received
0926 11:40:41.894983 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:41.894985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:41.895034 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:41.895170 (+ 136us) server_negotiation.cc:226] Negotiated authn=JWT
0926 11:40:41.895407 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:41.895944 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:41.897215 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:41.897440 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:41.897525 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:41.897847 (+ 322us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 11:40:41.909769 (+ 11922us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38133/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 11:40:41.910015 (+ 246us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52692: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38133/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":48}
W20260926 11:40:41.910198 11808 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52692: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38133/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260926 11:40:41.910399 11834 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:41.894381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45851 (local address 127.0.0.1:52692)
0926 11:40:41.894717 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:41.894732 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:41.894869 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:41.895176 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:41.895179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:41.895187 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0926 11:40:41.895315 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:41.895321 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:41.896087 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:41.896091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:41.897093 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:41.897104 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:41.897533 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:41.897555 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:41.897646 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:41.897677 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 11:40:41.909987 (+ 12310us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38133/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 11:40:41.910219 (+ 232us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.96.62:45851: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38133/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":80,"threads_started":1}
I20260926 11:40:41.911070 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 11738
2026-09-26T11:40:41Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (310 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260926 11:40:41.949150 6528 mini_oidc.cc:152] Starting JWKS server
I20260926 11:40:41.950035 6528 webserver.cc:460] Webserver started at https://127.0.0.1:33309/ using document root <none> and password file <none>
I20260926 11:40:41.950139 6528 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 11:40:41.950397 6528 webserver.cc:460] Webserver started at http://127.0.0.1:42539/ using document root <none> and password file <none>
2026-09-26T11:40:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:40:41Z Disabled control of system clock
I20260926 11:40:41.958468 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:45641
--webserver_interface=127.6.96.62
--webserver_port=0
--builtin_ntp_servers=127.6.96.20:33769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:45641
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33309/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 11:40:42.065891 11841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:42.066169 11841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:42.066207 11841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:42.070178 11841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:40:42.070241 11841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:42.070255 11841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:42.070271 11841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:40:42.070284 11841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 11:40:42.070299 11841 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 11:40:42.070318 11841 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33309/jwks.json
I20260926 11:40:42.074986 11841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:33769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:45641
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:45641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33309/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:42.075994 11841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:42.077298 11841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:42.085400 11847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:42.085393 11846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:42.085491 11841 server_base.cc:1034] running on GCE node
W20260926 11:40:42.085393 11849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:42.086165 11841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:42.087298 11841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:42.088443 11841 hybrid_clock.cc:648] HybridClock initialized: now 1790422842088419 us; error 38 us; skew 500 ppm
I20260926 11:40:42.090243 11841 webserver.cc:460] Webserver started at http://127.6.96.62:40137/ using document root <none> and password file <none>
I20260926 11:40:42.090785 11841 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:42.090878 11841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:42.091071 11841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:42.093030 11841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "0bde255ec90d489d8e67441c9012bed9"
format_stamp: "Formatted at 2026-09-26 11:40:42 on dist-test-slave-xxxx"
server_key: "0f05ebe36e194a3f68766fc4ec6d70fe"
server_key_iv: "a349068a47df8eeca4b1a3dd4eef9891"
server_key_version: "encryptionkey@0"
I20260926 11:40:42.093531 11841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "0bde255ec90d489d8e67441c9012bed9"
format_stamp: "Formatted at 2026-09-26 11:40:42 on dist-test-slave-xxxx"
server_key: "0f05ebe36e194a3f68766fc4ec6d70fe"
server_key_iv: "a349068a47df8eeca4b1a3dd4eef9891"
server_key_version: "encryptionkey@0"
I20260926 11:40:42.096972 11841 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260926 11:40:42.099258 11855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:42.100500 11841 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:40:42.100606 11841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "0bde255ec90d489d8e67441c9012bed9"
format_stamp: "Formatted at 2026-09-26 11:40:42 on dist-test-slave-xxxx"
server_key: "0f05ebe36e194a3f68766fc4ec6d70fe"
server_key_iv: "a349068a47df8eeca4b1a3dd4eef9891"
server_key_version: "encryptionkey@0"
I20260926 11:40:42.100701 11841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:42.135300 11841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:42.135996 11841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:42.136175 11841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:42.142797 11841 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:45641
I20260926 11:40:42.142904 11907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:45641 every 8 connection(s)
I20260926 11:40:42.143780 11841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:40:42.147082 11908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:42.152866 11908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bde255ec90d489d8e67441c9012bed9: Bootstrap starting.
I20260926 11:40:42.154085 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 11841
I20260926 11:40:42.154217 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:40:42.154498 6528 external_mini_cluster.cc:1428] Setting key 252fc1c944336015425c45eec6475ad4
I20260926 11:40:42.155925 11908 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bde255ec90d489d8e67441c9012bed9: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:42.156669 11908 log.cc:824] T 00000000000000000000000000000000 P 0bde255ec90d489d8e67441c9012bed9: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:42.158762 11908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bde255ec90d489d8e67441c9012bed9: No bootstrap required, opened a new log
I20260926 11:40:42.162762 11908 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bde255ec90d489d8e67441c9012bed9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bde255ec90d489d8e67441c9012bed9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45641 } }
I20260926 11:40:42.163040 11908 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bde255ec90d489d8e67441c9012bed9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:42.163077 11908 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bde255ec90d489d8e67441c9012bed9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bde255ec90d489d8e67441c9012bed9, State: Initialized, Role: FOLLOWER
I20260926 11:40:42.163424 11911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:42.155605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52532 (local address 127.6.96.62:45641)
0926 11:40:42.156245 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:42.156260 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:42.156296 (+ 36us) server_negotiation.cc:407] Connection header received
0926 11:40:42.156961 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:42.156984 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:42.157408 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:42.157744 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:42.158075 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:42.159079 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:42.160219 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:42.160685 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:42.160795 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:42.161591 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:42.161596 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:42.161609 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:40:42.161988 (+ 379us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:42.162043 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:40:42.162064 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:42.162135 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:42.162265 (+ 130us) server_negotiation.cc:299] Negotiation successful
0926 11:40:42.162524 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":124,"threads_started":1}
I20260926 11:40:42.163511 11908 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bde255ec90d489d8e67441c9012bed9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bde255ec90d489d8e67441c9012bed9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45641 } }
I20260926 11:40:42.163656 11908 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bde255ec90d489d8e67441c9012bed9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:40:42.163698 11908 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bde255ec90d489d8e67441c9012bed9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:40:42.163759 11908 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bde255ec90d489d8e67441c9012bed9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:42.164592 11908 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bde255ec90d489d8e67441c9012bed9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bde255ec90d489d8e67441c9012bed9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45641 } }
I20260926 11:40:42.164908 11908 leader_election.cc:302] T 00000000000000000000000000000000 P 0bde255ec90d489d8e67441c9012bed9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bde255ec90d489d8e67441c9012bed9; no voters:
I20260926 11:40:42.165198 11908 leader_election.cc:288] T 00000000000000000000000000000000 P 0bde255ec90d489d8e67441c9012bed9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:40:42.165547 11913 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bde255ec90d489d8e67441c9012bed9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:42.165707 11913 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bde255ec90d489d8e67441c9012bed9 [term 1 LEADER]: Becoming Leader. State: Replica: 0bde255ec90d489d8e67441c9012bed9, State: Running, Role: LEADER
I20260926 11:40:42.166004 11913 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bde255ec90d489d8e67441c9012bed9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bde255ec90d489d8e67441c9012bed9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45641 } }
I20260926 11:40:42.166107 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 11:40:42.166538 11908 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bde255ec90d489d8e67441c9012bed9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:40:42.167579 11915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bde255ec90d489d8e67441c9012bed9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bde255ec90d489d8e67441c9012bed9. Latest consensus state: current_term: 1 leader_uuid: "0bde255ec90d489d8e67441c9012bed9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bde255ec90d489d8e67441c9012bed9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45641 } } }
I20260926 11:40:42.167699 11915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bde255ec90d489d8e67441c9012bed9 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:42.168020 11914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bde255ec90d489d8e67441c9012bed9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bde255ec90d489d8e67441c9012bed9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bde255ec90d489d8e67441c9012bed9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45641 } } }
I20260926 11:40:42.168073 11922 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:40:42.168124 11914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bde255ec90d489d8e67441c9012bed9 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:42.171164 11922 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:40:42.176923 11922 catalog_manager.cc:1347] Generated new cluster ID: 0edbabc62e954d07a32b88f1f08da55c
I20260926 11:40:42.177014 11922 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:40:42.195688 11922 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:40:42.196628 11922 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:40:42.201205 11922 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bde255ec90d489d8e67441c9012bed9: Generated new TSK 0
I20260926 11:40:42.202026 11922 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:40:42.218448 11911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:42.213803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52542 (local address 127.6.96.62:45641)
0926 11:40:42.213918 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:42.213923 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:42.214069 (+ 146us) server_negotiation.cc:407] Connection header received
0926 11:40:42.214231 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:42.214234 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:42.214279 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:42.214371 (+ 92us) server_negotiation.cc:226] Negotiated authn=JWT
0926 11:40:42.214599 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:42.215330 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:42.216332 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:42.216565 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:42.216650 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:42.216947 (+ 297us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 11:40:42.218096 (+ 1149us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33309/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 11:40:42.218309 (+ 213us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52542: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33309/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":40}
W20260926 11:40:42.218617 11911 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52542: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33309/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260926 11:40:42.218962 11937 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:42.213734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45641 (local address 127.0.0.1:52542)
0926 11:40:42.213991 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:42.214004 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:42.214111 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:42.214378 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:42.214381 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:42.214389 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0926 11:40:42.214492 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:42.214499 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:42.215454 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:42.215458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:42.216222 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:42.216232 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:42.216656 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:42.216678 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:42.216765 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:42.216787 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 11:40:42.218237 (+ 1450us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33309/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 11:40:42.218375 (+ 138us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.96.62:45641: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33309/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":95,"threads_started":1}
I20260926 11:40:42.219504 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 11841
E20260926 11:40:42.231372 11938 webserver.cc:533] Webserver: sslize failed:
E20260926 11:40:42.231451 11938 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-26T11:40:42Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (309 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-26T11:40:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:40:42Z Disabled control of system clock
I20260926 11:40:42.266449 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:40101
--webserver_interface=127.6.96.62
--webserver_port=0
--builtin_ntp_servers=127.6.96.20:41265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:40101
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790422817500949-6528-0/testchainca.cert with env {}
W20260926 11:40:42.374573 11942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:42.374878 11942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:42.374927 11942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:42.378880 11942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:40:42.378939 11942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:42.378953 11942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:42.378968 11942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:40:42.378983 11942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 11:40:42.378997 11942 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 11:40:42.379009 11942 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260926 11:40:42.384368 11942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:41265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:40101
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:40101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790422817500949-6528-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:42.385353 11942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:42.386269 11942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:42.394507 11947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:42.394485 11950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:42.394529 11942 server_base.cc:1034] running on GCE node
W20260926 11:40:42.394491 11948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:42.395277 11942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:42.396348 11942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:42.397519 11942 hybrid_clock.cc:648] HybridClock initialized: now 1790422842397494 us; error 39 us; skew 500 ppm
I20260926 11:40:42.399435 11942 webserver.cc:460] Webserver started at http://127.6.96.62:33371/ using document root <none> and password file <none>
I20260926 11:40:42.399992 11942 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:42.400080 11942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:42.400272 11942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:42.402323 11942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "0b7a2536748f4d64b8ba8705707f70dd"
format_stamp: "Formatted at 2026-09-26 11:40:42 on dist-test-slave-xxxx"
server_key: "f941158ef12058bdef16a2f48b2efa64"
server_key_iv: "b4ff634d9e87cd355e2025419d854db6"
server_key_version: "encryptionkey@0"
I20260926 11:40:42.402887 11942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "0b7a2536748f4d64b8ba8705707f70dd"
format_stamp: "Formatted at 2026-09-26 11:40:42 on dist-test-slave-xxxx"
server_key: "f941158ef12058bdef16a2f48b2efa64"
server_key_iv: "b4ff634d9e87cd355e2025419d854db6"
server_key_version: "encryptionkey@0"
I20260926 11:40:42.406441 11942 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:40:42.408821 11956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:42.410029 11942 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:42.410161 11942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "0b7a2536748f4d64b8ba8705707f70dd"
format_stamp: "Formatted at 2026-09-26 11:40:42 on dist-test-slave-xxxx"
server_key: "f941158ef12058bdef16a2f48b2efa64"
server_key_iv: "b4ff634d9e87cd355e2025419d854db6"
server_key_version: "encryptionkey@0"
I20260926 11:40:42.410254 11942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:42.431006 11942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:42.431720 11942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:42.431891 11942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:42.438792 11942 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:40101
I20260926 11:40:42.438802 12008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:40101 every 8 connection(s)
I20260926 11:40:42.439801 11942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:40:42.441811 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 11942
I20260926 11:40:42.441931 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:40:42.442206 6528 external_mini_cluster.cc:1428] Setting key d36b3fa4db0a7297c53c88dea104d04e
I20260926 11:40:42.443902 12009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:42.449290 12009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b7a2536748f4d64b8ba8705707f70dd: Bootstrap starting.
I20260926 11:40:42.451140 12012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:42.443429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48686 (local address 127.6.96.62:40101)
0926 11:40:42.444022 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:42.444038 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:42.444074 (+ 36us) server_negotiation.cc:407] Connection header received
0926 11:40:42.444697 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:42.444713 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:42.445106 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:42.445409 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:42.445820 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:42.446777 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:42.447672 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:42.448144 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:42.448306 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:42.449249 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:42.449253 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:42.449266 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:40:42.449819 (+ 553us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:42.449875 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:40:42.449895 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:42.449994 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:42.450132 (+ 138us) server_negotiation.cc:299] Negotiation successful
0926 11:40:42.450375 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":101,"threads_started":1}
I20260926 11:40:42.451562 12009 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b7a2536748f4d64b8ba8705707f70dd: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:42.452306 12009 log.cc:824] T 00000000000000000000000000000000 P 0b7a2536748f4d64b8ba8705707f70dd: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:42.453809 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 11:40:42.454314 12009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b7a2536748f4d64b8ba8705707f70dd: No bootstrap required, opened a new log
I20260926 11:40:42.457685 12009 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b7a2536748f4d64b8ba8705707f70dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b7a2536748f4d64b8ba8705707f70dd" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40101 } }
I20260926 11:40:42.457898 12009 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b7a2536748f4d64b8ba8705707f70dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:42.457960 12009 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b7a2536748f4d64b8ba8705707f70dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b7a2536748f4d64b8ba8705707f70dd, State: Initialized, Role: FOLLOWER
I20260926 11:40:42.458359 12009 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b7a2536748f4d64b8ba8705707f70dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b7a2536748f4d64b8ba8705707f70dd" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40101 } }
I20260926 11:40:42.458490 12009 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b7a2536748f4d64b8ba8705707f70dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:40:42.458552 12009 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b7a2536748f4d64b8ba8705707f70dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:40:42.458644 12009 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b7a2536748f4d64b8ba8705707f70dd [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:42.459524 12009 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b7a2536748f4d64b8ba8705707f70dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b7a2536748f4d64b8ba8705707f70dd" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40101 } }
I20260926 11:40:42.459861 12009 leader_election.cc:302] T 00000000000000000000000000000000 P 0b7a2536748f4d64b8ba8705707f70dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b7a2536748f4d64b8ba8705707f70dd; no voters:
I20260926 11:40:42.460168 12009 leader_election.cc:288] T 00000000000000000000000000000000 P 0b7a2536748f4d64b8ba8705707f70dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:40:42.460265 12014 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b7a2536748f4d64b8ba8705707f70dd [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:42.460489 12014 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b7a2536748f4d64b8ba8705707f70dd [term 1 LEADER]: Becoming Leader. State: Replica: 0b7a2536748f4d64b8ba8705707f70dd, State: Running, Role: LEADER
I20260926 11:40:42.460822 12014 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b7a2536748f4d64b8ba8705707f70dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b7a2536748f4d64b8ba8705707f70dd" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40101 } }
I20260926 11:40:42.461290 12009 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b7a2536748f4d64b8ba8705707f70dd [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:40:42.462397 12015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b7a2536748f4d64b8ba8705707f70dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b7a2536748f4d64b8ba8705707f70dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b7a2536748f4d64b8ba8705707f70dd" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40101 } } }
I20260926 11:40:42.462443 12016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b7a2536748f4d64b8ba8705707f70dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b7a2536748f4d64b8ba8705707f70dd. Latest consensus state: current_term: 1 leader_uuid: "0b7a2536748f4d64b8ba8705707f70dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b7a2536748f4d64b8ba8705707f70dd" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40101 } } }
I20260926 11:40:42.462523 12015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b7a2536748f4d64b8ba8705707f70dd [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:42.462531 12016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b7a2536748f4d64b8ba8705707f70dd [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:42.462960 12023 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:40:42.465601 12023 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:40:42.470722 12023 catalog_manager.cc:1347] Generated new cluster ID: a99a623288af483cbaf75fd8026c0098
I20260926 11:40:42.470801 12023 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:40:42.490360 12023 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:40:42.491382 12023 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:40:42.499446 12023 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b7a2536748f4d64b8ba8705707f70dd: Generated new TSK 0
I20260926 11:40:42.500324 12023 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:40:42.604732 12012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:42.519718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48690 (local address 127.6.96.62:40101)
0926 11:40:42.519850 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:42.519855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:42.520050 (+ 195us) server_negotiation.cc:407] Connection header received
0926 11:40:42.520262 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:42.520265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:42.520314 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:42.520408 (+ 94us) server_negotiation.cc:226] Negotiated authn=JWT
0926 11:40:42.520645 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:42.521165 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:42.522010 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:42.522224 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:42.522309 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:42.522622 (+ 313us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 11:40:42.604270 (+ 81648us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 11:40:42.604566 (+ 296us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48690: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":50}
W20260926 11:40:42.604823 12012 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48690: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260926 11:40:42.604851 12038 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:42.519623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:40101 (local address 127.0.0.1:48690)
0926 11:40:42.519970 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:42.519981 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:42.520124 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:42.520426 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:42.520429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:42.520437 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0926 11:40:42.520548 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:42.520554 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:42.521277 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:42.521282 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:42.521904 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:42.521911 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:42.522314 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:42.522340 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:42.522432 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:42.522456 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 11:40:42.604493 (+ 82037us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 11:40:42.604678 (+ 185us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.96.62:40101: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":97,"threads_started":1}
I20260926 11:40:42.605566 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 11942
2026-09-26T11:40:42Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (364 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (121 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (119 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:40:42 dist-test-slave-xxxx krb5kdc[12050](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:40:42 dist-test-slave-xxxx krb5kdc[12050](info): set up 2 sockets
Sep 26 11:40:42 dist-test-slave-xxxx krb5kdc[12050](info): commencing operation
krb5kdc: starting...
W20260926 11:40:43.806809 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:40:43 dist-test-slave-xxxx krb5kdc[12050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:40:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:40:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:40:43.921360 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:44767
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:34881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:44767
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:44.026594 12063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:44.026885 12063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:44.026926 12063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:44.030886 12063 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:40:44.030949 12063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:44.030968 12063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:44.030983 12063 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:40:44.030997 12063 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:40:44.036283 12063 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:34881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:44767
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:44767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:44.037298 12063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:44.038262 12063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:44.046416 12068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:44.046411 12071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:44.046466 12063 server_base.cc:1034] running on GCE node
W20260926 11:40:44.046478 12069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:44.047214 12063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:44.048228 12063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:44.049377 12063 hybrid_clock.cc:648] HybridClock initialized: now 1790422844049358 us; error 33 us; skew 500 ppm
Sep 26 11:40:44 dist-test-slave-xxxx krb5kdc[12050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422844, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:44.052006 12063 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:40:44.052918 12063 webserver.cc:460] Webserver started at http://127.6.96.62:35271/ using document root <none> and password file <none>
I20260926 11:40:44.053428 12063 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:44.053490 12063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:44.053675 12063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:44.055686 12063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "eaac633d05714edc8346e99dcc830d66"
format_stamp: "Formatted at 2026-09-26 11:40:44 on dist-test-slave-xxxx"
server_key: "d5f311e9fb2fbb25d111abc715159f11"
server_key_iv: "0b6e8af32d15baea48f0e3c24238ab46"
server_key_version: "encryptionkey@0"
I20260926 11:40:44.056226 12063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "eaac633d05714edc8346e99dcc830d66"
format_stamp: "Formatted at 2026-09-26 11:40:44 on dist-test-slave-xxxx"
server_key: "d5f311e9fb2fbb25d111abc715159f11"
server_key_iv: "0b6e8af32d15baea48f0e3c24238ab46"
server_key_version: "encryptionkey@0"
I20260926 11:40:44.059731 12063 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:40:44.061930 12078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:44.063148 12063 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:40:44.063256 12063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "eaac633d05714edc8346e99dcc830d66"
format_stamp: "Formatted at 2026-09-26 11:40:44 on dist-test-slave-xxxx"
server_key: "d5f311e9fb2fbb25d111abc715159f11"
server_key_iv: "0b6e8af32d15baea48f0e3c24238ab46"
server_key_version: "encryptionkey@0"
I20260926 11:40:44.063349 12063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:44.074344 12063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:44.077239 12063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:44.077406 12063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:44.083743 12063 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:44767
I20260926 11:40:44.083771 12130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:44767 every 8 connection(s)
I20260926 11:40:44.084630 12063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:40:44.086748 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 12063
I20260926 11:40:44.086866 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:40:44.087126 6528 external_mini_cluster.cc:1428] Setting key ffd93bc3d105910ffb3b81ed3f3fb53b
I20260926 11:40:44.087435 12131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:44.094357 12131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaac633d05714edc8346e99dcc830d66: Bootstrap starting.
Sep 26 11:40:44 dist-test-slave-xxxx krb5kdc[12050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:44.096722 12131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eaac633d05714edc8346e99dcc830d66: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:44.097640 12131 log.cc:824] T 00000000000000000000000000000000 P eaac633d05714edc8346e99dcc830d66: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:44.099895 12131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaac633d05714edc8346e99dcc830d66: No bootstrap required, opened a new log
I20260926 11:40:44.101379 12134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:44.088492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58424 (local address 127.6.96.62:44767)
0926 11:40:44.088899 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:44.088918 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:44.088959 (+ 41us) server_negotiation.cc:407] Connection header received
0926 11:40:44.089652 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:44.089674 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:44.090112 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:44.090478 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:44.091178 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:44.092587 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:44.093398 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:44.094071 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:44.094234 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:44.096484 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:44.096490 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:44.096506 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:44.096538 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:44.098297 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:44.098748 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:44.098754 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:44.098760 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:44.098918 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:44.099211 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:44.099214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:44.099216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:44.099722 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:44.099906 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:44.100125 (+ 219us) server_negotiation.cc:299] Negotiation successful
0926 11:40:44.100431 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":112,"threads_started":1}
I20260926 11:40:44.104079 12131 raft_consensus.cc:348] T 00000000000000000000000000000000 P eaac633d05714edc8346e99dcc830d66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaac633d05714edc8346e99dcc830d66" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44767 } }
I20260926 11:40:44.104305 12131 raft_consensus.cc:374] T 00000000000000000000000000000000 P eaac633d05714edc8346e99dcc830d66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:44.104349 12131 raft_consensus.cc:729] T 00000000000000000000000000000000 P eaac633d05714edc8346e99dcc830d66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaac633d05714edc8346e99dcc830d66, State: Initialized, Role: FOLLOWER
I20260926 11:40:44.104707 12131 consensus_queue.cc:260] T 00000000000000000000000000000000 P eaac633d05714edc8346e99dcc830d66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaac633d05714edc8346e99dcc830d66" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44767 } }
I20260926 11:40:44.104815 12131 raft_consensus.cc:388] T 00000000000000000000000000000000 P eaac633d05714edc8346e99dcc830d66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:40:44.104856 12131 raft_consensus.cc:482] T 00000000000000000000000000000000 P eaac633d05714edc8346e99dcc830d66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:40:44.104921 12131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eaac633d05714edc8346e99dcc830d66 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:44.105746 12131 raft_consensus.cc:504] T 00000000000000000000000000000000 P eaac633d05714edc8346e99dcc830d66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaac633d05714edc8346e99dcc830d66" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44767 } }
I20260926 11:40:44.106030 12131 leader_election.cc:302] T 00000000000000000000000000000000 P eaac633d05714edc8346e99dcc830d66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eaac633d05714edc8346e99dcc830d66; no voters:
I20260926 11:40:44.106294 12131 leader_election.cc:288] T 00000000000000000000000000000000 P eaac633d05714edc8346e99dcc830d66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:40:44.106398 12136 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eaac633d05714edc8346e99dcc830d66 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:44.106593 12136 raft_consensus.cc:686] T 00000000000000000000000000000000 P eaac633d05714edc8346e99dcc830d66 [term 1 LEADER]: Becoming Leader. State: Replica: eaac633d05714edc8346e99dcc830d66, State: Running, Role: LEADER
I20260926 11:40:44.106943 12136 consensus_queue.cc:237] T 00000000000000000000000000000000 P eaac633d05714edc8346e99dcc830d66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaac633d05714edc8346e99dcc830d66" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44767 } }
I20260926 11:40:44.107514 12131 sys_catalog.cc:564] T 00000000000000000000000000000000 P eaac633d05714edc8346e99dcc830d66 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:40:44.108680 12137 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaac633d05714edc8346e99dcc830d66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eaac633d05714edc8346e99dcc830d66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaac633d05714edc8346e99dcc830d66" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44767 } } }
I20260926 11:40:44.108829 12137 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaac633d05714edc8346e99dcc830d66 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:44.108664 12138 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaac633d05714edc8346e99dcc830d66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eaac633d05714edc8346e99dcc830d66. Latest consensus state: current_term: 1 leader_uuid: "eaac633d05714edc8346e99dcc830d66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaac633d05714edc8346e99dcc830d66" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44767 } } }
I20260926 11:40:44.109079 12138 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaac633d05714edc8346e99dcc830d66 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:44.109192 12145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:40:44.111994 12145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:40:44.117637 12145 catalog_manager.cc:1347] Generated new cluster ID: 36ddfbf5b67648088b7690a39a1a40fb
I20260926 11:40:44.117724 12145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:40:44.138115 12145 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:40:44.139143 12145 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:40:44.149115 12145 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eaac633d05714edc8346e99dcc830d66: Generated new TSK 0
I20260926 11:40:44.149830 12145 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.96.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
I20260926 11:40:44.182260 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:44767
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:34881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:44.290891 12157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:44.291157 12157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:44.291199 12157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:44.295519 12157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:44.295588 12157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:44.295655 12157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:40:44.300913 12157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:34881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:44767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:44.301942 12157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:44.302984 12157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:44.312189 12162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:44.312194 12165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:44.312315 12163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:44.312408 12157 server_base.cc:1034] running on GCE node
I20260926 11:40:44.312891 12157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:44.313530 12157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:44.314687 12157 hybrid_clock.cc:648] HybridClock initialized: now 1790422844314667 us; error 31 us; skew 500 ppm
Sep 26 11:40:44 dist-test-slave-xxxx krb5kdc[12050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422844, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:44.317382 12157 init.cc:377] Logged in from keytab as kudu/127.6.96.1@KRBTEST.COM (short username kudu)
I20260926 11:40:44.318274 12157 webserver.cc:460] Webserver started at http://127.6.96.1:39683/ using document root <none> and password file <none>
I20260926 11:40:44.318796 12157 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:44.318897 12157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:44.319106 12157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:44.321151 12157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "6b63d5757e764359b532c3413b1c0867"
format_stamp: "Formatted at 2026-09-26 11:40:44 on dist-test-slave-xxxx"
server_key: "3a201eb3e06f389b9500fcab27b539a6"
server_key_iv: "ecd7a9c3f96f20ca2c55519f151c5e55"
server_key_version: "encryptionkey@0"
I20260926 11:40:44.321703 12157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "6b63d5757e764359b532c3413b1c0867"
format_stamp: "Formatted at 2026-09-26 11:40:44 on dist-test-slave-xxxx"
server_key: "3a201eb3e06f389b9500fcab27b539a6"
server_key_iv: "ecd7a9c3f96f20ca2c55519f151c5e55"
server_key_version: "encryptionkey@0"
I20260926 11:40:44.325402 12157 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260926 11:40:44.327824 12172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:44.329023 12157 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:40:44.329125 12157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "6b63d5757e764359b532c3413b1c0867"
format_stamp: "Formatted at 2026-09-26 11:40:44 on dist-test-slave-xxxx"
server_key: "3a201eb3e06f389b9500fcab27b539a6"
server_key_iv: "ecd7a9c3f96f20ca2c55519f151c5e55"
server_key_version: "encryptionkey@0"
I20260926 11:40:44.329234 12157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:44.341842 12157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:44.344626 12157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:44.344801 12157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:44.345351 12157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:44.346128 12157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:44.346174 12157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:44.346235 12157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:44.346261 12157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:44.354578 12157 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:45017
I20260926 11:40:44.354601 12285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:45017 every 8 connection(s)
I20260926 11:40:44.355441 12157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:40:44.358398 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 12157
I20260926 11:40:44.358528 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:40:44.358860 6528 external_mini_cluster.cc:1428] Setting key 100a3499ca4512b1bf2ad6810d9f138c
Sep 26 11:40:44 dist-test-slave-xxxx krb5kdc[12050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422844, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.96.2@KRBTEST.COM; defaulting to no policy
I20260926 11:40:44.369762 12134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:44.357139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:41633 (local address 127.6.96.62:44767)
0926 11:40:44.357293 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:44.357298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:44.358110 (+ 812us) server_negotiation.cc:407] Connection header received
0926 11:40:44.359811 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:44.359815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:44.359860 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:44.359906 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:44.361541 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:44.362097 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:44.363280 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:44.363622 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:44.363717 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:44.366053 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:44.366057 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:44.366060 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:44.366098 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:44.367737 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:44.368232 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:44.368236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:44.368238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:44.368324 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:44.368712 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:44.368714 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:44.368715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:44.368890 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:44.368965 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:44.369524 (+ 559us) server_negotiation.cc:299] Negotiation successful
0926 11:40:44.369637 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 11:40:44.370607 12288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:44.357532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44767 (local address 127.6.96.1:41633)
0926 11:40:44.357964 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:44.358004 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:44.359548 (+ 1544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:44.360269 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:44.360276 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:44.360732 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:44.361392 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:44.361410 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:44.362226 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:44.362228 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:44.363168 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:44.363178 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:44.363756 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:44.363759 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:44.363909 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:44.364442 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:44.364457 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:44.365891 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:44.367850 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:44.367855 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:44.367867 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:44.368130 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:44.368443 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:44.368447 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:44.368450 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:44.368620 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:44.368979 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:44.368984 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:44.369249 (+ 265us) client_negotiation.cc:769] Sending connection context
0926 11:40:44.369409 (+ 160us) client_negotiation.cc:240] Negotiation successful
0926 11:40:44.369712 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":111,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.2@KRBTEST.COM" created.
I20260926 11:40:44.371801 12286 heartbeater.cc:347] Connected to a master server at 127.6.96.62:44767
I20260926 11:40:44.372051 12286 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:44.372562 12286 heartbeater.cc:511] Master 127.6.96.62:44767 requested a full tablet report, sending...
I20260926 11:40:44.374128 12095 ts_manager.cc:194] Registered new tserver with Master: 6b63d5757e764359b532c3413b1c0867 (127.6.96.1:45017)
I20260926 11:40:44.375705 12095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.1@KRBTEST.COM'} at 127.6.96.1:41633
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
I20260926 11:40:44.385547 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:44767
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:34881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:44.491801 12291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:44.492065 12291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:44.492110 12291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:44.496126 12291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:44.496186 12291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:44.496245 12291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:40:44.501420 12291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:34881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:44767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:44.502373 12291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:44.503347 12291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:44.512766 12296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:44.512709 12297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:44.512720 12299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:44.512867 12291 server_base.cc:1034] running on GCE node
I20260926 11:40:44.513411 12291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:44.514027 12291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:44.515170 12291 hybrid_clock.cc:648] HybridClock initialized: now 1790422844515151 us; error 31 us; skew 500 ppm
Sep 26 11:40:44 dist-test-slave-xxxx krb5kdc[12050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422844, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:44.517866 12291 init.cc:377] Logged in from keytab as kudu/127.6.96.2@KRBTEST.COM (short username kudu)
I20260926 11:40:44.518759 12291 webserver.cc:460] Webserver started at http://127.6.96.2:42613/ using document root <none> and password file <none>
I20260926 11:40:44.519330 12291 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:44.519395 12291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:44.519580 12291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:44.521667 12291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "bcc6e8ceeb564155afdf1ed0c9d1fa04"
format_stamp: "Formatted at 2026-09-26 11:40:44 on dist-test-slave-xxxx"
server_key: "8ead4fd9126280ffc8e074f16b1d9654"
server_key_iv: "9575bfce2e75ed1b2bf507d9af5ca9b6"
server_key_version: "encryptionkey@0"
I20260926 11:40:44.522229 12291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "bcc6e8ceeb564155afdf1ed0c9d1fa04"
format_stamp: "Formatted at 2026-09-26 11:40:44 on dist-test-slave-xxxx"
server_key: "8ead4fd9126280ffc8e074f16b1d9654"
server_key_iv: "9575bfce2e75ed1b2bf507d9af5ca9b6"
server_key_version: "encryptionkey@0"
I20260926 11:40:44.526079 12291 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 11:40:44.528631 12306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:44.529932 12291 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 11:40:44.530056 12291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "bcc6e8ceeb564155afdf1ed0c9d1fa04"
format_stamp: "Formatted at 2026-09-26 11:40:44 on dist-test-slave-xxxx"
server_key: "8ead4fd9126280ffc8e074f16b1d9654"
server_key_iv: "9575bfce2e75ed1b2bf507d9af5ca9b6"
server_key_version: "encryptionkey@0"
I20260926 11:40:44.530159 12291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:44.543828 12291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:44.546589 12291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:44.546772 12291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:44.547363 12291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:44.548169 12291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:44.548224 12291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:44.548277 12291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:44.548301 12291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:44.556720 12291 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:37221
I20260926 11:40:44.556746 12419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:37221 every 8 connection(s)
I20260926 11:40:44.557582 12291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:40:44.561129 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 12291
I20260926 11:40:44.561275 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:40:44.561550 6528 external_mini_cluster.cc:1428] Setting key a48765f33848aad5e2ca5edb4137bc7e
Sep 26 11:40:44 dist-test-slave-xxxx krb5kdc[12050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422844, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.96.3@KRBTEST.COM; defaulting to no policy
I20260926 11:40:44.572046 12134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:44.559367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:53025 (local address 127.6.96.62:44767)
0926 11:40:44.559515 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:44.559520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:44.560411 (+ 891us) server_negotiation.cc:407] Connection header received
0926 11:40:44.562053 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:44.562057 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:44.562113 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:44.562186 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:44.563585 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:44.564163 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:44.565389 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:44.565699 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:44.565805 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:44.568398 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:44.568402 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:44.568405 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:44.568435 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:44.570048 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:44.570529 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:44.570532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:44.570534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:44.570628 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:44.571007 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:44.571010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:44.571011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:44.571187 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:44.571266 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:44.571817 (+ 551us) server_negotiation.cc:299] Negotiation successful
0926 11:40:44.571918 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.3@KRBTEST.COM" created.
I20260926 11:40:44.572948 12422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:44.559764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44767 (local address 127.6.96.2:53025)
0926 11:40:44.560249 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:44.560293 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:44.561806 (+ 1513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:44.562377 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:44.562385 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:44.562754 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:44.563442 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:44.563462 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:44.564299 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:44.564303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:44.565275 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:44.565284 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:44.565861 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:44.565865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:44.566014 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:44.566690 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:44.566707 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:44.568228 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:44.570162 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:44.570167 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:44.570180 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:44.570431 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:44.570741 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:44.570743 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:44.570745 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:44.570924 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:44.571311 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:44.571319 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:44.571602 (+ 283us) client_negotiation.cc:769] Sending connection context
0926 11:40:44.571767 (+ 165us) client_negotiation.cc:240] Negotiation successful
0926 11:40:44.572039 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":112,"threads_started":1}
I20260926 11:40:44.575470 12420 heartbeater.cc:347] Connected to a master server at 127.6.96.62:44767
I20260926 11:40:44.575719 12420 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:44.576207 12420 heartbeater.cc:511] Master 127.6.96.62:44767 requested a full tablet report, sending...
I20260926 11:40:44.577296 12095 ts_manager.cc:194] Registered new tserver with Master: bcc6e8ceeb564155afdf1ed0c9d1fa04 (127.6.96.2:37221)
I20260926 11:40:44.578053 12095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.2@KRBTEST.COM'} at 127.6.96.2:53025
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
I20260926 11:40:44.589485 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:44767
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:34881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:44.696686 12425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:44.696954 12425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:44.696995 12425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:44.700932 12425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:44.700986 12425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:44.701042 12425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:40:44.706069 12425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:34881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:44767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:44.707055 12425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:44.708045 12425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:44.717509 12431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:44.717592 12433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:44.717862 12430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:44.720491 12425 server_base.cc:1034] running on GCE node
I20260926 11:40:44.720985 12425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:44.721709 12425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:44.722891 12425 hybrid_clock.cc:648] HybridClock initialized: now 1790422844722856 us; error 54 us; skew 500 ppm
Sep 26 11:40:44 dist-test-slave-xxxx krb5kdc[12050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422844, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:44.726212 12425 init.cc:377] Logged in from keytab as kudu/127.6.96.3@KRBTEST.COM (short username kudu)
I20260926 11:40:44.727097 12425 webserver.cc:460] Webserver started at http://127.6.96.3:46415/ using document root <none> and password file <none>
I20260926 11:40:44.727612 12425 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:44.727667 12425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:44.727842 12425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:44.729808 12425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "5d921d742175428787297f8836ef4ce6"
format_stamp: "Formatted at 2026-09-26 11:40:44 on dist-test-slave-xxxx"
server_key: "bbcbd2d8e0877c32f22f7d2ad4023e1b"
server_key_iv: "e69f1e00a931f152d8ec214a7560d3d3"
server_key_version: "encryptionkey@0"
I20260926 11:40:44.730324 12425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "5d921d742175428787297f8836ef4ce6"
format_stamp: "Formatted at 2026-09-26 11:40:44 on dist-test-slave-xxxx"
server_key: "bbcbd2d8e0877c32f22f7d2ad4023e1b"
server_key_iv: "e69f1e00a931f152d8ec214a7560d3d3"
server_key_version: "encryptionkey@0"
I20260926 11:40:44.733938 12425 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 11:40:44.736351 12440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:44.737515 12425 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 11:40:44.737622 12425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "5d921d742175428787297f8836ef4ce6"
format_stamp: "Formatted at 2026-09-26 11:40:44 on dist-test-slave-xxxx"
server_key: "bbcbd2d8e0877c32f22f7d2ad4023e1b"
server_key_iv: "e69f1e00a931f152d8ec214a7560d3d3"
server_key_version: "encryptionkey@0"
I20260926 11:40:44.737715 12425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:44.756244 12425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:44.759318 12425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:44.759505 12425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:44.760103 12425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:44.760931 12425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:44.760977 12425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:44.761027 12425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:44.761065 12425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:44.769536 12425 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:33257
I20260926 11:40:44.769567 12553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:33257 every 8 connection(s)
I20260926 11:40:44.770421 12425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:40:44.775341 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 12425
I20260926 11:40:44.775439 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:40:44.775662 6528 external_mini_cluster.cc:1428] Setting key 91e1f8f2caad5618d8055700fe281431
Sep 26 11:40:44 dist-test-slave-xxxx krb5kdc[12050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422844, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:44.783275 12134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:44.772197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:47217 (local address 127.6.96.62:44767)
0926 11:40:44.772375 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:44.772380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:44.773244 (+ 864us) server_negotiation.cc:407] Connection header received
0926 11:40:44.774546 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:44.774549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:44.774598 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:44.774674 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:44.775931 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:44.776581 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:44.777718 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:44.777932 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:44.778016 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:44.780267 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:44.780271 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:44.780273 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:44.780295 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:44.781436 (+ 1141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:44.781881 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:44.781884 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:44.781887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:44.781959 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:44.782251 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:44.782254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:44.782255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:44.782418 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:44.782498 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:44.783003 (+ 505us) server_negotiation.cc:299] Negotiation successful
0926 11:40:44.783115 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 11:40:44.784011 12556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:44.772605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44767 (local address 127.6.96.3:47217)
0926 11:40:44.773090 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:44.773131 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:44.774348 (+ 1217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:44.774884 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:44.774892 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:44.775260 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:44.775791 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:44.775808 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:44.776694 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:44.776698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:44.777598 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:44.777607 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:44.778057 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:44.778061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:44.778227 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:44.778746 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:44.778761 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:44.780115 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:44.781535 (+ 1420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:44.781541 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:44.781555 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:44.781795 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:44.782036 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:44.782039 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:44.782041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:44.782162 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:44.782509 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:44.782516 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:44.782757 (+ 241us) client_negotiation.cc:769] Sending connection context
0926 11:40:44.782941 (+ 184us) client_negotiation.cc:240] Negotiation successful
0926 11:40:44.783254 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":115,"threads_started":1}
I20260926 11:40:44.784986 12554 heartbeater.cc:347] Connected to a master server at 127.6.96.62:44767
I20260926 11:40:44.785177 12554 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:44.785588 12554 heartbeater.cc:511] Master 127.6.96.62:44767 requested a full tablet report, sending...
I20260926 11:40:44.786475 12095 ts_manager.cc:194] Registered new tserver with Master: 5d921d742175428787297f8836ef4ce6 (127.6.96.3:33257)
I20260926 11:40:44.787047 12095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.3@KRBTEST.COM'} at 127.6.96.3:47217
I20260926 11:40:44.788914 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/test-admin.keytab.
Sep 26 11:40:44 dist-test-slave-xxxx krb5kdc[12050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:44.803507 6528 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260926 11:40:44.806205 12561 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:44.804851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58854 (local address 127.6.96.1:45017)
0926 11:40:44.805151 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:44.805163 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:44.805189 (+ 26us) server_negotiation.cc:407] Connection header received
0926 11:40:44.805239 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:44.805243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:44.805430 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:44.805558 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:44.806034 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58854: BlockingRecv error: recv got EOF from 127.0.0.1:58854 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":87,"threads_started":1}
W20260926 11:40:44.806309 6528 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/krb5cc' of type FILE
Sep 26 11:40:44 dist-test-slave-xxxx krb5kdc[12050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:44.807402 6528 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 11:40:44 dist-test-slave-xxxx krb5kdc[12050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:44.808359 6528 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 11:40:44 dist-test-slave-xxxx krb5kdc[12050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.1@KRBTEST.COM
I20260926 11:40:44.817687 12561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:44.809183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58856 (local address 127.6.96.1:45017)
0926 11:40:44.809303 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:44.809308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:44.809371 (+ 63us) server_negotiation.cc:407] Connection header received
0926 11:40:44.809481 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:44.809483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:44.809521 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:44.809585 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:44.810020 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:44.810872 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:44.811660 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:44.812039 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:44.812114 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:44.814202 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:44.814212 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:44.814227 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:44.814252 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:44.815826 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:44.816304 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:44.816310 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:44.816311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:44.816386 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:44.816726 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:44.816729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:44.816730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:44.816984 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:44.817129 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:44.817384 (+ 255us) server_negotiation.cc:299] Negotiation successful
0926 11:40:44.817514 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260926 11:40:44.821509 12561 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:44.820588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58868 (local address 127.6.96.1:45017)
0926 11:40:44.820706 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:44.820710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:44.820807 (+ 97us) server_negotiation.cc:407] Connection header received
0926 11:40:44.820921 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:44.820924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:44.820960 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:44.821021 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:44.821413 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58868: BlockingRecv error: recv got EOF from 127.0.0.1:58868 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260926 11:40:44.821624 6528 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/krb5cc' of type FILE
Sep 26 11:40:44 dist-test-slave-xxxx krb5kdc[12050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:44.822570 6528 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 11:40:44 dist-test-slave-xxxx krb5kdc[12050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:44.823382 6528 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 11:40:44 dist-test-slave-xxxx krb5kdc[12050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.1@KRBTEST.COM
I20260926 11:40:44.831667 12561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:44.824279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58870 (local address 127.6.96.1:45017)
0926 11:40:44.824388 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:44.824392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:44.824405 (+ 13us) server_negotiation.cc:407] Connection header received
0926 11:40:44.824543 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:44.824546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:44.824584 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:44.824656 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:44.825159 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:44.825684 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:44.826483 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:44.826726 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:44.826807 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:44.828890 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:44.828893 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:44.828895 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:44.828916 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:44.830234 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:44.830587 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:44.830590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:44.830591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:44.830667 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:44.830905 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:44.830909 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:44.830911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:44.831080 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:44.831194 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:44.831413 (+ 219us) server_negotiation.cc:299] Negotiation successful
0926 11:40:44.831523 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260926 11:40:44.834272 12561 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:44.833265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58878 (local address 127.6.96.1:45017)
0926 11:40:44.833370 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:44.833374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:44.833562 (+ 188us) server_negotiation.cc:407] Connection header received
0926 11:40:44.833683 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:44.833686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:44.833726 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:44.833788 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:44.834141 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58878: BlockingRecv error: recv got EOF from 127.0.0.1:58878 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260926 11:40:44.834668 6528 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790422817500949-6528-0/krb5kdc/krb5cc)
Sep 26 11:40:44 dist-test-slave-xxxx krb5kdc[12050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:44.835656 6528 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 11:40:44 dist-test-slave-xxxx krb5kdc[12050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:44.836464 6528 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 11:40:44 dist-test-slave-xxxx krb5kdc[12050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.1@KRBTEST.COM
I20260926 11:40:44.843791 12561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:44.837194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58882 (local address 127.6.96.1:45017)
0926 11:40:44.837312 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:44.837315 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:44.837384 (+ 69us) server_negotiation.cc:407] Connection header received
0926 11:40:44.837472 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:44.837474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:44.837503 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:44.837562 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:44.837960 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:44.838474 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:44.839233 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:44.839436 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:44.839509 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:44.841356 (+ 1847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:44.841359 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:44.841361 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:44.841380 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:44.842635 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:44.842990 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:44.842993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:44.842994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:44.843057 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:44.843226 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:44.843228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:44.843229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:44.843357 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:44.843438 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:44.843576 (+ 138us) server_negotiation.cc:299] Negotiation successful
0926 11:40:44.843653 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 11:40:44.844760 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 12157
I20260926 11:40:44.850008 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 12291
I20260926 11:40:44.854588 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 12425
I20260926 11:40:44.858662 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 12063
2026-09-26T11:40:44Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2014 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:40:44 dist-test-slave-xxxx krb5kdc[12574](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:40:44 dist-test-slave-xxxx krb5kdc[12574](info): set up 2 sockets
Sep 26 11:40:44 dist-test-slave-xxxx krb5kdc[12574](info): commencing operation
krb5kdc: starting...
W20260926 11:40:45.811090 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:40:45 dist-test-slave-xxxx krb5kdc[12574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:40:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:40:45Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.62.keytab.
Entry for principal oryx/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.62.keytab.
I20260926 11:40:45.929970 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:44995
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.62.keytab
--principal=oryx/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:42279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:44995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:46.040073 12587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:46.040377 12587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:46.040446 12587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:46.044585 12587 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:40:46.044669 12587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:46.044698 12587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:46.044718 12587 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:40:46.044737 12587 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:40:46.050294 12587 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:42279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:44995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.62.keytab
--principal=oryx/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:44995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:46.051501 12587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:46.052572 12587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:46.061439 12592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:46.061523 12593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:46.061653 12587 server_base.cc:1034] running on GCE node
W20260926 11:40:46.061439 12595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:46.062638 12587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:46.063915 12587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:46.065099 12587 hybrid_clock.cc:648] HybridClock initialized: now 1790422846065086 us; error 29 us; skew 500 ppm
Sep 26 11:40:46 dist-test-slave-xxxx krb5kdc[12574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422846, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:46.067888 12587 init.cc:377] Logged in from keytab as oryx/127.6.96.62@KRBTEST.COM (short username oryx)
I20260926 11:40:46.068872 12587 webserver.cc:460] Webserver started at http://127.6.96.62:38853/ using document root <none> and password file <none>
I20260926 11:40:46.069446 12587 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:46.069509 12587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:46.069705 12587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:46.071854 12587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "10fbe49d79eb4051ab5b16c45aba3cc7"
format_stamp: "Formatted at 2026-09-26 11:40:46 on dist-test-slave-xxxx"
server_key: "3eb0fac68237c9f5599bc0312c8352fd"
server_key_iv: "5929e0c5d5893b87913ceeabf1c34898"
server_key_version: "encryptionkey@0"
I20260926 11:40:46.072408 12587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "10fbe49d79eb4051ab5b16c45aba3cc7"
format_stamp: "Formatted at 2026-09-26 11:40:46 on dist-test-slave-xxxx"
server_key: "3eb0fac68237c9f5599bc0312c8352fd"
server_key_iv: "5929e0c5d5893b87913ceeabf1c34898"
server_key_version: "encryptionkey@0"
I20260926 11:40:46.076378 12587 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 11:40:46.079043 12602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:46.080480 12587 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:40:46.080613 12587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "10fbe49d79eb4051ab5b16c45aba3cc7"
format_stamp: "Formatted at 2026-09-26 11:40:46 on dist-test-slave-xxxx"
server_key: "3eb0fac68237c9f5599bc0312c8352fd"
server_key_iv: "5929e0c5d5893b87913ceeabf1c34898"
server_key_version: "encryptionkey@0"
I20260926 11:40:46.080736 12587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:46.092181 12587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:46.095391 12587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:46.095563 12587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:46.102522 12587 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:44995
I20260926 11:40:46.102787 12654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:44995 every 8 connection(s)
I20260926 11:40:46.103567 12587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:40:46.106156 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 12587
I20260926 11:40:46.106266 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:40:46.106472 12655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:46.106544 6528 external_mini_cluster.cc:1428] Setting key 149ad0eca81de3df73b1ea1b06a978d7
I20260926 11:40:46.113665 12655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7: Bootstrap starting.
Sep 26 11:40:46 dist-test-slave-xxxx krb5kdc[12574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM
I20260926 11:40:46.116493 12655 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:46.117404 12655 log.cc:824] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:46.119757 12655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7: No bootstrap required, opened a new log
I20260926 11:40:46.121918 12658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:46.107974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38202 (local address 127.6.96.62:44995)
0926 11:40:46.108377 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:46.108395 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:46.108439 (+ 44us) server_negotiation.cc:407] Connection header received
0926 11:40:46.109178 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:46.109202 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:46.109652 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:46.110036 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:46.110705 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:46.112188 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:46.113088 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:46.113773 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:46.113923 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:46.116556 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:46.116563 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:46.116581 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:46.116616 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:46.118471 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:46.118987 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:46.118994 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:46.119000 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:46.119134 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:46.119422 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:46.119426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:46.119427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:46.119945 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:46.120131 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:46.120409 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 11:40:46.120727 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":107,"threads_started":1}
I20260926 11:40:46.123649 12655 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } }
I20260926 11:40:46.123891 12655 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:46.123957 12655 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10fbe49d79eb4051ab5b16c45aba3cc7, State: Initialized, Role: FOLLOWER
I20260926 11:40:46.124433 12655 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } }
I20260926 11:40:46.124570 12655 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:40:46.124634 12655 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:40:46.124732 12655 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:46.125654 12655 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } }
I20260926 11:40:46.126001 12655 leader_election.cc:302] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10fbe49d79eb4051ab5b16c45aba3cc7; no voters:
I20260926 11:40:46.126312 12655 leader_election.cc:288] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:40:46.126432 12660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:46.127027 12660 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [term 1 LEADER]: Becoming Leader. State: Replica: 10fbe49d79eb4051ab5b16c45aba3cc7, State: Running, Role: LEADER
I20260926 11:40:46.127569 12655 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:40:46.127594 12660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } }
I20260926 11:40:46.128682 12660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10fbe49d79eb4051ab5b16c45aba3cc7. Latest consensus state: current_term: 1 leader_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } }
I20260926 11:40:46.128866 12660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:46.129060 12661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } }
I20260926 11:40:46.129156 12661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:46.129408 12669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:40:46.132206 12669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:40:46.137530 12669 catalog_manager.cc:1347] Generated new cluster ID: ab48db96286a4772873e443a7a2f1221
I20260926 11:40:46.137595 12669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:40:46.154659 12669 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:40:46.155769 12669 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:40:46.162631 12669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7: Generated new TSK 0
I20260926 11:40:46.163391 12669 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.96.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.96.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.96.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.1.keytab.
Entry for principal oryx/127.6.96.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.1.keytab.
I20260926 11:40:46.190755 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:44995
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.1.keytab
--principal=oryx/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:42279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:46.302191 12681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:46.302492 12681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:46.302537 12681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:46.306625 12681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:46.306701 12681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:46.306770 12681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:40:46.312500 12681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:42279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.1.keytab
--principal=oryx/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:44995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:46.313614 12681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:46.314658 12681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:46.324231 12686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:46.324327 12689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:46.324272 12687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:46.324409 12681 server_base.cc:1034] running on GCE node
I20260926 11:40:46.324894 12681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:46.325523 12681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:46.326709 12681 hybrid_clock.cc:648] HybridClock initialized: now 1790422846326688 us; error 32 us; skew 500 ppm
Sep 26 11:40:46 dist-test-slave-xxxx krb5kdc[12574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422846, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:46.329555 12681 init.cc:377] Logged in from keytab as oryx/127.6.96.1@KRBTEST.COM (short username oryx)
I20260926 11:40:46.330448 12681 webserver.cc:460] Webserver started at http://127.6.96.1:39091/ using document root <none> and password file <none>
I20260926 11:40:46.331027 12681 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:46.331096 12681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:46.331310 12681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:46.333302 12681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "10f2ba5d863c469c9084ea7824990d8d"
format_stamp: "Formatted at 2026-09-26 11:40:46 on dist-test-slave-xxxx"
server_key: "cb864365836311b5f0ac80c35ec4ecf4"
server_key_iv: "ff075e36e9f1cee7255572e8f44e88c2"
server_key_version: "encryptionkey@0"
I20260926 11:40:46.333833 12681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "10f2ba5d863c469c9084ea7824990d8d"
format_stamp: "Formatted at 2026-09-26 11:40:46 on dist-test-slave-xxxx"
server_key: "cb864365836311b5f0ac80c35ec4ecf4"
server_key_iv: "ff075e36e9f1cee7255572e8f44e88c2"
server_key_version: "encryptionkey@0"
I20260926 11:40:46.337491 12681 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 11:40:46.339835 12696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:46.341060 12681 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:40:46.341166 12681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "10f2ba5d863c469c9084ea7824990d8d"
format_stamp: "Formatted at 2026-09-26 11:40:46 on dist-test-slave-xxxx"
server_key: "cb864365836311b5f0ac80c35ec4ecf4"
server_key_iv: "ff075e36e9f1cee7255572e8f44e88c2"
server_key_version: "encryptionkey@0"
I20260926 11:40:46.341259 12681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:46.355338 12681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:46.358388 12681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:46.358563 12681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:46.359185 12681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:46.360001 12681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:46.360047 12681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:46.360098 12681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:46.360126 12681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:46.368628 12681 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:38253
I20260926 11:40:46.368650 12809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:38253 every 8 connection(s)
I20260926 11:40:46.369521 12681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:40:46.377295 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 12681
I20260926 11:40:46.377409 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:40:46.377701 6528 external_mini_cluster.cc:1428] Setting key e1ac694fa9493b9fda86aae974eec6de
Sep 26 11:40:46 dist-test-slave-xxxx krb5kdc[12574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422846, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM
I20260926 11:40:46.382936 12658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:46.371378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:33215 (local address 127.6.96.62:44995)
0926 11:40:46.371517 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:46.371522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:46.372650 (+ 1128us) server_negotiation.cc:407] Connection header received
0926 11:40:46.373536 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:46.373539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:46.373588 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:46.373674 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:46.374904 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:46.375475 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:46.376583 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:46.376824 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:46.376916 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:46.379508 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:46.379512 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:46.379514 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:46.379540 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:46.380868 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:46.381384 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:46.381387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:46.381389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:46.381476 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:46.381790 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:46.381793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:46.381795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:46.381980 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:46.382066 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:46.382602 (+ 536us) server_negotiation.cc:299] Negotiation successful
0926 11:40:46.382731 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 11:40:46.383710 12812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:46.371797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.6.96.1:33215)
0926 11:40:46.372493 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:46.372536 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:46.373315 (+ 779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:46.373869 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:46.373877 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:46.374228 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:46.374728 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:46.374741 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:46.375571 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:46.375574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:46.376450 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:46.376458 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:46.376964 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:46.376967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:46.377111 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:46.377694 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:46.377711 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:46.379355 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:46.380991 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:46.380997 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:46.381013 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:46.381285 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:46.381559 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:46.381562 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:46.381564 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:46.381704 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:46.382080 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:46.382087 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:46.382338 (+ 251us) client_negotiation.cc:769] Sending connection context
0926 11:40:46.382524 (+ 186us) client_negotiation.cc:240] Negotiation successful
0926 11:40:46.382811 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":117,"threads_started":1}
I20260926 11:40:46.384898 12810 heartbeater.cc:347] Connected to a master server at 127.6.96.62:44995
I20260926 11:40:46.385156 12810 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:46.385725 12810 heartbeater.cc:511] Master 127.6.96.62:44995 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.6.96.2@KRBTEST.COM; defaulting to no policy
I20260926 11:40:46.387426 12619 ts_manager.cc:194] Registered new tserver with Master: 10f2ba5d863c469c9084ea7824990d8d (127.6.96.1:38253)
I20260926 11:40:46.388566 12619 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.96.1@KRBTEST.COM'} at 127.6.96.1:33215
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.96.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.96.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.2.keytab.
Entry for principal oryx/127.6.96.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.2.keytab.
I20260926 11:40:46.403162 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:44995
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.2.keytab
--principal=oryx/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:42279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:46.523377 12815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:46.523643 12815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:46.523762 12815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:46.528087 12815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:46.528156 12815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:46.528235 12815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:40:46.533746 12815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:42279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.2.keytab
--principal=oryx/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:44995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:46.534781 12815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:46.535811 12815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:46.545181 12820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:46.545162 12823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:46.545183 12821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:46.545324 12815 server_base.cc:1034] running on GCE node
I20260926 11:40:46.545797 12815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:46.546439 12815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:46.547585 12815 hybrid_clock.cc:648] HybridClock initialized: now 1790422846547568 us; error 28 us; skew 500 ppm
Sep 26 11:40:46 dist-test-slave-xxxx krb5kdc[12574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422846, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:46.550069 12815 init.cc:377] Logged in from keytab as oryx/127.6.96.2@KRBTEST.COM (short username oryx)
I20260926 11:40:46.550976 12815 webserver.cc:460] Webserver started at http://127.6.96.2:42047/ using document root <none> and password file <none>
I20260926 11:40:46.551481 12815 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:46.551535 12815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:46.551692 12815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:46.553632 12815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "ec00c3f1615d44b6adeedbe52d77f35c"
format_stamp: "Formatted at 2026-09-26 11:40:46 on dist-test-slave-xxxx"
server_key: "e6f81715ce6570948c03ca8694239223"
server_key_iv: "4d58726f50de4a0264382f3403251f9b"
server_key_version: "encryptionkey@0"
I20260926 11:40:46.554132 12815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "ec00c3f1615d44b6adeedbe52d77f35c"
format_stamp: "Formatted at 2026-09-26 11:40:46 on dist-test-slave-xxxx"
server_key: "e6f81715ce6570948c03ca8694239223"
server_key_iv: "4d58726f50de4a0264382f3403251f9b"
server_key_version: "encryptionkey@0"
I20260926 11:40:46.557667 12815 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 11:40:46.560024 12830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:46.561239 12815 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 11:40:46.561357 12815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "ec00c3f1615d44b6adeedbe52d77f35c"
format_stamp: "Formatted at 2026-09-26 11:40:46 on dist-test-slave-xxxx"
server_key: "e6f81715ce6570948c03ca8694239223"
server_key_iv: "4d58726f50de4a0264382f3403251f9b"
server_key_version: "encryptionkey@0"
I20260926 11:40:46.561446 12815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:46.585119 12815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:46.588042 12815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:46.588205 12815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:46.588783 12815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:46.589594 12815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:46.589638 12815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:46.589679 12815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:46.589707 12815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:46.597852 12815 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:46263
I20260926 11:40:46.597890 12943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:46263 every 8 connection(s)
I20260926 11:40:46.598734 12815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:40:46.599088 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 12815
I20260926 11:40:46.599191 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:40:46.599524 6528 external_mini_cluster.cc:1428] Setting key ccd23d3fe44f5abea629e0acbe09b809
WARNING: no policy specified for oryx/127.6.96.3@KRBTEST.COM; defaulting to no policy
Sep 26 11:40:46 dist-test-slave-xxxx krb5kdc[12574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422846, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.96.3@KRBTEST.COM" created.
I20260926 11:40:46.612236 12658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:46.600507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:36099 (local address 127.6.96.62:44995)
0926 11:40:46.600634 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:46.600639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:46.601752 (+ 1113us) server_negotiation.cc:407] Connection header received
0926 11:40:46.602612 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:46.602615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:46.602667 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:46.602742 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:46.604117 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:46.604711 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:46.605985 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:46.606207 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:46.606293 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:46.608814 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:46.608818 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:46.608821 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:46.608846 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:46.610143 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:46.610631 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:46.610634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:46.610635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:46.610707 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:46.611078 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:46.611081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:46.611082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:46.611280 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:46.611359 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:46.611997 (+ 638us) server_negotiation.cc:299] Negotiation successful
0926 11:40:46.612105 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 11:40:46.613145 12946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:46.600951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.6.96.2:36099)
0926 11:40:46.601620 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:46.601660 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:46.602419 (+ 759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:46.602963 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:46.602970 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:46.603324 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:46.603968 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:46.603988 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:46.604863 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:46.604866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:46.605867 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:46.605878 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:46.606343 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:46.606347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:46.606512 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:46.607092 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:46.607108 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:46.608641 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:46.610243 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:46.610248 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:46.610261 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:46.610508 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:46.610805 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:46.610808 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:46.610811 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:46.610976 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:46.611383 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:46.611389 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:46.611709 (+ 320us) client_negotiation.cc:769] Sending connection context
0926 11:40:46.611905 (+ 196us) client_negotiation.cc:240] Negotiation successful
0926 11:40:46.612269 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":135,"threads_started":1}
I20260926 11:40:46.614249 12944 heartbeater.cc:347] Connected to a master server at 127.6.96.62:44995
I20260926 11:40:46.614521 12944 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:46.615134 12944 heartbeater.cc:511] Master 127.6.96.62:44995 requested a full tablet report, sending...
I20260926 11:40:46.616294 12619 ts_manager.cc:194] Registered new tserver with Master: ec00c3f1615d44b6adeedbe52d77f35c (127.6.96.2:46263)
I20260926 11:40:46.617054 12619 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.96.2@KRBTEST.COM'} at 127.6.96.2:36099
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.96.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.3.keytab.
Entry for principal oryx/127.6.96.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.3.keytab.
I20260926 11:40:46.625785 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:44995
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.3.keytab
--principal=oryx/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:42279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:46.734072 12949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:46.734341 12949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:46.734392 12949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:46.738366 12949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:46.738436 12949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:46.738497 12949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:40:46.743701 12949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:42279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.3.keytab
--principal=oryx/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:44995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:46.744728 12949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:46.745723 12949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:46.754922 12954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:46.754925 12957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:46.754963 12955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:46.754961 12949 server_base.cc:1034] running on GCE node
I20260926 11:40:46.755566 12949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:46.756199 12949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:46.757354 12949 hybrid_clock.cc:648] HybridClock initialized: now 1790422846757330 us; error 40 us; skew 500 ppm
Sep 26 11:40:46 dist-test-slave-xxxx krb5kdc[12574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422846, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:46.759943 12949 init.cc:377] Logged in from keytab as oryx/127.6.96.3@KRBTEST.COM (short username oryx)
I20260926 11:40:46.760785 12949 webserver.cc:460] Webserver started at http://127.6.96.3:44513/ using document root <none> and password file <none>
I20260926 11:40:46.761305 12949 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:46.761365 12949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:46.761530 12949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:46.763540 12949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "c8733268bb584adbb5deeaac640b5e3f"
format_stamp: "Formatted at 2026-09-26 11:40:46 on dist-test-slave-xxxx"
server_key: "8521467f1534b0c488d9001c04a44477"
server_key_iv: "8b326471796eeefdbfcc4cd03a15d999"
server_key_version: "encryptionkey@0"
I20260926 11:40:46.764077 12949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "c8733268bb584adbb5deeaac640b5e3f"
format_stamp: "Formatted at 2026-09-26 11:40:46 on dist-test-slave-xxxx"
server_key: "8521467f1534b0c488d9001c04a44477"
server_key_iv: "8b326471796eeefdbfcc4cd03a15d999"
server_key_version: "encryptionkey@0"
I20260926 11:40:46.767773 12949 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:40:46.769997 12964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:46.771221 12949 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:40:46.771320 12949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "c8733268bb584adbb5deeaac640b5e3f"
format_stamp: "Formatted at 2026-09-26 11:40:46 on dist-test-slave-xxxx"
server_key: "8521467f1534b0c488d9001c04a44477"
server_key_iv: "8b326471796eeefdbfcc4cd03a15d999"
server_key_version: "encryptionkey@0"
I20260926 11:40:46.771402 12949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:46.780387 12949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:46.783215 12949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:46.783360 12949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:46.783921 12949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:46.784693 12949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:46.784737 12949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:46.784775 12949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:46.784793 12949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:46.792966 12949 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:37879
I20260926 11:40:46.793015 13077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:37879 every 8 connection(s)
I20260926 11:40:46.793839 12949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:40:46.801656 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 12949
I20260926 11:40:46.801774 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:40:46.802047 6528 external_mini_cluster.cc:1428] Setting key af0b6c553f1e9aeea2f32a362e8e6e5d
Sep 26 11:40:46 dist-test-slave-xxxx krb5kdc[12574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422846, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM
I20260926 11:40:46.806387 12658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:46.795604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:37545 (local address 127.6.96.62:44995)
0926 11:40:46.795742 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:46.795746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:46.796700 (+ 954us) server_negotiation.cc:407] Connection header received
0926 11:40:46.797524 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:46.797526 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:46.797569 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:46.797639 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:46.798739 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:46.799260 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:46.800343 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:46.800553 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:46.800619 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:46.803146 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:46.803149 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:46.803151 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:46.803175 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:46.804572 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:46.805050 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:46.805053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:46.805054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:46.805133 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:46.805434 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:46.805436 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:46.805437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:46.805607 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:46.805676 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:46.806159 (+ 483us) server_negotiation.cc:299] Negotiation successful
0926 11:40:46.806261 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 11:40:46.807294 13080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:46.795910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.6.96.3:37545)
0926 11:40:46.796559 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:46.796601 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:46.797356 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:46.797802 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:46.797809 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:46.798156 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:46.798629 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:46.798641 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:46.799363 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:46.799366 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:46.800251 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:46.800260 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:46.800662 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:46.800664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:46.800816 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:46.801317 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:46.801334 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:46.802993 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:46.804687 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:46.804691 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:46.804705 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:46.804960 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:46.805216 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:46.805219 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:46.805220 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:46.805365 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:46.805696 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:46.805701 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:46.805945 (+ 244us) client_negotiation.cc:769] Sending connection context
0926 11:40:46.806103 (+ 158us) client_negotiation.cc:240] Negotiation successful
0926 11:40:46.806357 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":105,"threads_started":1}
I20260926 11:40:46.808326 13078 heartbeater.cc:347] Connected to a master server at 127.6.96.62:44995
I20260926 11:40:46.808544 13078 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:46.809008 13078 heartbeater.cc:511] Master 127.6.96.62:44995 requested a full tablet report, sending...
I20260926 11:40:46.810015 12619 ts_manager.cc:194] Registered new tserver with Master: c8733268bb584adbb5deeaac640b5e3f (127.6.96.3:37879)
I20260926 11:40:46.810587 12619 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.96.3@KRBTEST.COM'} at 127.6.96.3:37545
I20260926 11:40:46.815892 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.96.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.96.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.96.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.61.keytab.
Entry for principal oryx/127.6.96.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.61.keytab.
I20260926 11:40:46.841714 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.61:39733
--webserver_interface=127.6.96.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.61.keytab
--principal=oryx/127.6.96.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:42279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:44995,127.6.96.61:39733
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:46.949506 13083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:46.949761 13083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:46.949808 13083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:46.953850 13083 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:40:46.953905 13083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:46.953924 13083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:46.953939 13083 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:40:46.953953 13083 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:40:46.959231 13083 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:42279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:44995,127.6.96.61:39733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.61.keytab
--principal=oryx/127.6.96.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.61:39733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.96.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
W20260926 11:40:46.959431 13083 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.96.62:44995,127.6.96.61:39733'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260926 11:40:46.960263 13083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:46.961259 13083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 11:40:46.969281 13083 server_base.cc:1034] running on GCE node
W20260926 11:40:46.969238 13088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:46.969245 13091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:46.969257 13089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:46.970120 13083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:46.971273 13083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:46.972429 13083 hybrid_clock.cc:648] HybridClock initialized: now 1790422846972411 us; error 35 us; skew 500 ppm
Sep 26 11:40:46 dist-test-slave-xxxx krb5kdc[12574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422846, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:46.974996 13083 init.cc:377] Logged in from keytab as oryx/127.6.96.61@KRBTEST.COM (short username oryx)
I20260926 11:40:46.975816 13083 webserver.cc:460] Webserver started at http://127.6.96.61:45387/ using document root <none> and password file <none>
I20260926 11:40:46.976315 13083 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:46.976374 13083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:46.976562 13083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:46.978514 13083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/data/instance:
uuid: "65753213026347eab7c588f0e98074c3"
format_stamp: "Formatted at 2026-09-26 11:40:46 on dist-test-slave-xxxx"
server_key: "085f546ff79b1c829b5efb92ef5acbd8"
server_key_iv: "f24dd1ebd0f85ededea81259fd73c8e5"
server_key_version: "encryptionkey@0"
I20260926 11:40:46.979064 13083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/wal/instance:
uuid: "65753213026347eab7c588f0e98074c3"
format_stamp: "Formatted at 2026-09-26 11:40:46 on dist-test-slave-xxxx"
server_key: "085f546ff79b1c829b5efb92ef5acbd8"
server_key_iv: "f24dd1ebd0f85ededea81259fd73c8e5"
server_key_version: "encryptionkey@0"
I20260926 11:40:46.982589 13083 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260926 11:40:46.985016 13098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:46.986227 13083 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:46.986323 13083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/wal
uuid: "65753213026347eab7c588f0e98074c3"
format_stamp: "Formatted at 2026-09-26 11:40:46 on dist-test-slave-xxxx"
server_key: "085f546ff79b1c829b5efb92ef5acbd8"
server_key_iv: "f24dd1ebd0f85ededea81259fd73c8e5"
server_key_version: "encryptionkey@0"
I20260926 11:40:46.986413 13083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:47.006765 13083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:47.009619 13083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:47.009780 13083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:47.016175 13083 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.61:39733
I20260926 11:40:47.016419 13150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.61:39733 every 8 connection(s)
I20260926 11:40:47.017124 13083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/data/info.pb
I20260926 11:40:47.017391 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 13083
I20260926 11:40:47.020193 13151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:47.023844 13151 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 11:40:47 dist-test-slave-xxxx krb5kdc[12574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.96.61@KRBTEST.COM
Sep 26 11:40:47 dist-test-slave-xxxx krb5kdc[12574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422846, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.61@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM
I20260926 11:40:47.031960 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.018952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34980 (local address 127.6.96.61:39733)
0926 11:40:47.019387 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:47.019402 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:47.019438 (+ 36us) server_negotiation.cc:407] Connection header received
0926 11:40:47.020120 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:47.020136 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:47.020492 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:47.020804 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:47.021555 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.022534 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.023449 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.023912 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.024050 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.026581 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:47.026586 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:47.026595 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:47.026622 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:47.028925 (+ 2303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.029383 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.029388 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.029395 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.029544 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.029806 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.029810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.029811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.030232 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:47.030420 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:47.030693 (+ 273us) server_negotiation.cc:299] Negotiation successful
0926 11:40:47.031020 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":148,"threads_started":1}
I20260926 11:40:47.034485 13155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.025513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.0.0.1:38212)
0926 11:40:47.025918 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:47.025966 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:47.026122 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:47.026456 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:47.026461 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:47.026802 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:47.026990 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.026998 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.027833 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.027837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.028894 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.028904 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.029458 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.029461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.029573 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.030028 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:47.030039 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:47.031409 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:47.033016 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.033024 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.033029 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.033227 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.033470 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.033473 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.033475 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.033548 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.033932 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:47.033938 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:47.034044 (+ 106us) client_negotiation.cc:769] Sending connection context
0926 11:40:47.034181 (+ 137us) client_negotiation.cc:240] Negotiation successful
0926 11:40:47.034329 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":120,"threads_started":1}
I20260926 11:40:47.034495 12658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.025619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38212 (local address 127.6.96.62:44995)
0926 11:40:47.025764 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:47.025769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:47.026038 (+ 269us) server_negotiation.cc:407] Connection header received
0926 11:40:47.026234 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:47.026238 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:47.026291 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:47.026380 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:47.027102 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.027673 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.029023 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.029344 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.029432 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.031540 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:47.031543 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:47.031545 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:47.031570 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:47.032917 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.033314 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.033318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.033319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.033392 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.033624 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.033628 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.033629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.033830 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:47.033922 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:47.034256 (+ 334us) server_negotiation.cc:299] Negotiation successful
0926 11:40:47.034384 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 11:40:47.035506 13151 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 11:40:47 dist-test-slave-xxxx krb5kdc[12574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422846, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.61@KRBTEST.COM for oryx/127.6.96.61@KRBTEST.COM
I20260926 11:40:47.042999 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.035985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34982 (local address 127.6.96.61:39733)
0926 11:40:47.036073 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:47.036077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:47.036089 (+ 12us) server_negotiation.cc:407] Connection header received
0926 11:40:47.036180 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:47.036183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:47.036218 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:47.036276 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:47.036612 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.037100 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.037817 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.038026 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.038107 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.040413 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:47.040417 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:47.040420 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:47.040440 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:47.041619 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.041965 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.041968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.041971 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.042031 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.042245 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.042249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.042250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.042409 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:47.042487 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:47.042700 (+ 213us) server_negotiation.cc:299] Negotiation successful
0926 11:40:47.042859 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":36}
I20260926 11:40:47.042999 13155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.035932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:39733 (local address 127.0.0.1:34982)
0926 11:40:47.036025 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:47.036038 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:47.036110 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:47.036302 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:47.036305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:47.036443 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:47.036521 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.036525 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.037187 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.037190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.037707 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.037713 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.038116 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.038119 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.038220 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.038625 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:47.038636 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:47.040269 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:47.041715 (+ 1446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.041718 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.041720 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.041889 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.042104 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.042106 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.042109 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.042176 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.042487 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:47.042490 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:47.042570 (+ 80us) client_negotiation.cc:769] Sending connection context
0926 11:40:47.042650 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 11:40:47.042793 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":3}
I20260926 11:40:47.046108 13151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3: Bootstrap starting.
I20260926 11:40:47.048435 13151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:47.049158 13151 log.cc:824] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:47.051112 13151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3: No bootstrap required, opened a new log
I20260926 11:40:47.054059 13151 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } }
I20260926 11:40:47.054298 13151 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:47.054335 13151 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65753213026347eab7c588f0e98074c3, State: Initialized, Role: FOLLOWER
I20260926 11:40:47.054774 13151 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } }
I20260926 11:40:47.055311 13157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } } }
I20260926 11:40:47.055434 13157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:40:47.056049 13151 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 11:40:47.060598 13168 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 11:40:47.060688 13168 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 11:40:47.062722 13083 master_runner.cc:428] Detected that this master 65753213026347eab7c588f0e98074c3 is joining an existing cluster
I20260926 11:40:47.062793 13083 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.96.61:39733
I20260926 11:40:47.073102 13175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.065059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:39733 (local address 127.0.0.1:34994)
0926 11:40:47.065588 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:47.065602 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:47.065709 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:47.066002 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:47.066005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:47.066263 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:47.066443 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.066449 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.067498 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.067501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.068088 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.068095 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.068521 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.068524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.068603 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.069707 (+ 1104us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:47.069719 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:47.070075 (+ 356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:47.071692 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.071695 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.071697 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.071878 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.072101 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.072103 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.072105 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.072176 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.072534 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:47.072536 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:47.072639 (+ 103us) client_negotiation.cc:769] Sending connection context
0926 11:40:47.072739 (+ 100us) client_negotiation.cc:240] Negotiation successful
0926 11:40:47.072881 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":16,"thread_start_us":95,"threads_started":1}
I20260926 11:40:47.073104 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.065196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34994 (local address 127.6.96.61:39733)
0926 11:40:47.065289 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:47.065293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:47.065686 (+ 393us) server_negotiation.cc:407] Connection header received
0926 11:40:47.065835 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:47.065839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:47.065885 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:47.065967 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:47.066906 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.067404 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.068183 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.068425 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.068504 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.070204 (+ 1700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:47.070207 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:47.070210 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:47.070228 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:47.071598 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.071954 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.071957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.071959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.072021 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.072244 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.072247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.072248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.072446 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:47.072539 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:47.072780 (+ 241us) server_negotiation.cc:299] Negotiation successful
0926 11:40:47.072944 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":35}
I20260926 11:40:47.073558 13174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.065059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.0.0.1:38220)
0926 11:40:47.065632 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:47.065642 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:47.065768 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:47.066122 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:47.066125 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:47.066324 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:47.066434 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.066442 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.067553 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.067557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.068509 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.068519 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.069014 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.069017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.069097 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.069490 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:47.069506 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:47.069967 (+ 461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:47.072027 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.072030 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.072032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.072265 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.072557 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.072560 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.072562 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.072632 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.073040 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:47.073043 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:47.073120 (+ 77us) client_negotiation.cc:769] Sending connection context
0926 11:40:47.073213 (+ 93us) client_negotiation.cc:240] Negotiation successful
0926 11:40:47.073408 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":266,"thread_start_us":117,"threads_started":1}
I20260926 11:40:47.073693 12658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.065141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38220 (local address 127.6.96.62:44995)
0926 11:40:47.065258 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:47.065263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:47.065700 (+ 437us) server_negotiation.cc:407] Connection header received
0926 11:40:47.065891 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:47.065894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:47.065945 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:47.066013 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:47.066550 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.067422 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.068652 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.068917 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.069004 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.070118 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:47.070123 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:47.070126 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:47.070168 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:47.071893 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.072378 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.072382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.072384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.072464 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.072711 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.072714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.072716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.072916 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:47.073002 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:47.073437 (+ 435us) server_negotiation.cc:299] Negotiation successful
0926 11:40:47.073550 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Sep 26 11:40:47 dist-test-slave-xxxx krb5kdc[12574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422846, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.62@KRBTEST.COM for oryx/127.6.96.61@KRBTEST.COM
I20260926 11:40:47.086313 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.077932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35008 (local address 127.6.96.61:39733)
0926 11:40:47.078027 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:47.078032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:47.078357 (+ 325us) server_negotiation.cc:407] Connection header received
0926 11:40:47.078531 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:47.078535 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:47.078577 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:47.078639 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:47.079382 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.079849 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.080690 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.080895 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.080975 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.083262 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:47.083265 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:47.083267 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:47.083288 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:47.084635 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.085184 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.085186 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.085187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.085255 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.085495 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.085498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.085500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.085662 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:47.085741 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:47.086054 (+ 313us) server_negotiation.cc:299] Negotiation successful
0926 11:40:47.086159 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 11:40:47.086335 13176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.077910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:39733 (local address 127.0.0.1:35008)
0926 11:40:47.078226 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:47.078283 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:47.078414 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:47.078684 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:47.078691 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:47.079154 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:47.079278 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.079284 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.079943 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.079947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.080590 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.080597 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.080983 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.080986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.081090 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.081555 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:47.081566 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:47.083140 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:47.084741 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.084751 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.084757 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.085005 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.085335 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.085338 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.085340 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.085421 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.085753 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:47.085760 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:47.085874 (+ 114us) client_negotiation.cc:769] Sending connection context
0926 11:40:47.085996 (+ 122us) client_negotiation.cc:240] Negotiation successful
0926 11:40:47.086183 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":86,"threads_started":1}
I20260926 11:40:47.087757 12619 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.96.61:39733: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "65753213026347eab7c588f0e98074c3"
member_type: NON_VOTER
last_known_addr {
host: "127.6.96.61"
port: 39733
}
attrs {
promote: true
}
}
dest_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7"
cas_config_opid_index: -1
I20260926 11:40:47.088449 12619 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: NON_VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: true } }
I20260926 11:40:47.090305 13178 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 65753213026347eab7c588f0e98074c3 (127.6.96.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: NON_VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: true } } }
I20260926 11:40:47.091302 13125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:47.092262 13178 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.96.61:39733
I20260926 11:40:47.092687 13125 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [term 1 FOLLOWER]: Refusing update from remote peer 10fbe49d79eb4051ab5b16c45aba3cc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260926 11:40:47.092725 13083 master.cc:537] Master@127.6.96.61:39733 shutting down...
WARNING: no policy specified for oryx/127.6.96.60@KRBTEST.COM; defaulting to no policy
I20260926 11:40:47.093654 13181 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: NON_VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:47.094571 13179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: NON_VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: true } } }
I20260926 11:40:47.094738 13179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [sys.catalog]: This master's current role is: LEADER
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.96.60@KRBTEST.COM" created.
W20260926 11:40:47.095991 12603 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260926 11:40:47.096814 12603 consensus_peers.cc:487] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 -> Peer 65753213026347eab7c588f0e98074c3 (127.6.96.61:39733): Couldn't send request to peer 65753213026347eab7c588f0e98074c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 11:40:47.097728 13083 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:47.097970 13083 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:47.098048 13083 tablet_replica.cc:330] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.96.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.60.keytab.
Entry for principal oryx/127.6.96.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.60.keytab.
I20260926 11:40:47.112670 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.60:35405
--webserver_interface=127.6.96.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.60.keytab
--principal=oryx/127.6.96.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:42279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:44995,127.6.96.61:39733,127.6.96.60:35405
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 11:40:47.113308 13083 master.cc:552] Master@127.6.96.61:39733 shutdown complete.
I20260926 11:40:47.113349 13083 master_runner.cc:305] Clearing existing system tablet
I20260926 11:40:47.113579 13083 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:47.113613 13083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:47.117333 13083 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 11:40:47.118259 13187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:47.118494 13083 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:47.118615 13083 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:47.118649 13083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/wal
uuid: "65753213026347eab7c588f0e98074c3"
format_stamp: "Formatted at 2026-09-26 11:40:46 on dist-test-slave-xxxx"
server_key: "085f546ff79b1c829b5efb92ef5acbd8"
server_key_iv: "f24dd1ebd0f85ededea81259fd73c8e5"
server_key_version: "encryptionkey@0"
I20260926 11:40:47.120893 13083 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:47.123152 13083 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 11:40:47.123237 13083 log.cc:1196] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260926 11:40:47.123481 13083 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3: Deleting consensus metadata
I20260926 11:40:47.123637 13083 master_runner.cc:315] Copying system tablet from 127.6.96.62:44995
I20260926 11:40:47.124766 13083 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3: tablet copy: Beginning tablet copy session from remote peer at address 127.6.96.62:44995
I20260926 11:40:47.134050 12658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.125399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38232 (local address 127.6.96.62:44995)
0926 11:40:47.125516 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:47.125521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:47.125694 (+ 173us) server_negotiation.cc:407] Connection header received
0926 11:40:47.125809 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:47.125812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:47.125863 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:47.125966 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:47.126428 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.127295 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.128594 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.128985 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.129082 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.130505 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:47.130509 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:47.130512 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:47.130538 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:47.132366 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.132855 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.132858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.132860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.132944 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.133196 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.133199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.133201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.133423 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:47.133515 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:47.133771 (+ 256us) server_negotiation.cc:299] Negotiation successful
0926 11:40:47.133889 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 11:40:47.134075 13193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.125304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.0.0.1:38232)
0926 11:40:47.125600 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:47.125614 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:47.125728 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:47.125989 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:47.125992 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:47.126194 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:47.126317 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.126324 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.127398 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.127401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.128482 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.128493 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.129148 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.129152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.129249 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.129908 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:47.129922 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:47.130359 (+ 437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:47.132486 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.132490 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.132492 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.132754 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.133027 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.133030 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.133032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.133119 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.133507 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:47.133510 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:47.133616 (+ 106us) client_negotiation.cc:769] Sending connection context
0926 11:40:47.133714 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 11:40:47.133869 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":87,"threads_started":1}
I20260926 11:40:47.135051 12639 tablet_copy_service.cc:140] P 10fbe49d79eb4051ab5b16c45aba3cc7: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 65753213026347eab7c588f0e98074c3 ({username='oryx', principal='oryx/127.6.96.61@KRBTEST.COM'} at 127.0.0.1:38232)
I20260926 11:40:47.135222 12639 tablet_copy_service.cc:161] P 10fbe49d79eb4051ab5b16c45aba3cc7: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 65753213026347eab7c588f0e98074c3 at {username='oryx', principal='oryx/127.6.96.61@KRBTEST.COM'} at 127.0.0.1:38232: session id = 65753213026347eab7c588f0e98074c3-00000000000000000000000000000000
I20260926 11:40:47.136878 12639 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 11:40:47.138504 13083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 11:40:47.141166 13083 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3: tablet copy: Starting download of 0 data blocks...
I20260926 11:40:47.141392 13083 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3: tablet copy: Starting download of 1 WAL segments...
I20260926 11:40:47.143184 13083 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 11:40:47.145223 12639 tablet_copy_service.cc:342] P 10fbe49d79eb4051ab5b16c45aba3cc7: Request end of tablet copy session 65753213026347eab7c588f0e98074c3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.96.61@KRBTEST.COM'} at 127.0.0.1:38232
I20260926 11:40:47.145412 12639 tablet_copy_service.cc:433] P 10fbe49d79eb4051ab5b16c45aba3cc7: ending tablet copy session 65753213026347eab7c588f0e98074c3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 65753213026347eab7c588f0e98074c3
I20260926 11:40:47.152518 13083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:47.154163 13194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:47.155874 13195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:47.157006 13083 server_base.cc:1034] running on GCE node
W20260926 11:40:47.158949 13197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:47.159170 13083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:47.159487 13083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:47.160632 13083 hybrid_clock.cc:648] HybridClock initialized: now 1790422847160621 us; error 30 us; skew 500 ppm
Sep 26 11:40:47 dist-test-slave-xxxx krb5kdc[12574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422847, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:47.162213 13083 init.cc:377] Logged in from keytab as oryx/127.6.96.61@KRBTEST.COM (short username oryx)
I20260926 11:40:47.162681 13083 webserver.cc:460] Webserver started at http://127.6.96.61:35913/ using document root <none> and password file <none>
I20260926 11:40:47.162942 13083 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:47.162989 13083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:47.164326 13083 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 11:40:47.165313 13204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:47.166010 13083 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 11:40:47.166085 13083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/wal
uuid: "65753213026347eab7c588f0e98074c3"
format_stamp: "Formatted at 2026-09-26 11:40:46 on dist-test-slave-xxxx"
server_key: "085f546ff79b1c829b5efb92ef5acbd8"
server_key_iv: "f24dd1ebd0f85ededea81259fd73c8e5"
server_key_version: "encryptionkey@0"
I20260926 11:40:47.166164 13083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:47.172505 13083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:47.174420 13083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:47.179319 13083 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.61:39733
I20260926 11:40:47.179342 13256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.61:39733 every 8 connection(s)
I20260926 11:40:47.179898 13083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-1/data/info.pb
I20260926 11:40:47.181600 13257 sys_catalog.cc:263] Verifying existing consensus state
I20260926 11:40:47.182603 13257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3: Bootstrap starting.
I20260926 11:40:47.192415 13257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 11:40:47.192688 13257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3: Bootstrap complete.
I20260926 11:40:47.193233 13257 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: NON_VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: true } }
I20260926 11:40:47.193362 13257 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 65753213026347eab7c588f0e98074c3, State: Initialized, Role: LEARNER
I20260926 11:40:47.193464 13257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: NON_VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: true } }
I20260926 11:40:47.193825 13260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: NON_VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: true } } }
I20260926 11:40:47.193903 13257 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:40:47.193992 13260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [sys.catalog]: This master's current role is: LEARNER
I20260926 11:40:47.196962 13271 catalog_manager.cc:1259] Loaded cluster ID: ab48db96286a4772873e443a7a2f1221
I20260926 11:40:47.197089 13271 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3: loading cluster ID for follower catalog manager: success
I20260926 11:40:47.199287 13271 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3: acquiring CA information for follower catalog manager: success
I20260926 11:40:47.200623 13271 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 11:40:47.243098 13185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:47.243381 13185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:47.243439 13185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:47.247591 13185 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:40:47.247661 13185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:47.247687 13185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:47.247710 13185 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:40:47.247740 13185 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:40:47.253151 13185 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:42279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:44995,127.6.96.61:39733,127.6.96.60:35405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.60.keytab
--principal=oryx/127.6.96.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.60:35405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.96.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:47.254238 13185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:47.255285 13185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:47.263381 13276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:47.263381 13279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:47.263455 13277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:47.263733 13185 server_base.cc:1034] running on GCE node
I20260926 11:40:47.264325 13185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:47.265460 13185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:47.266608 13185 hybrid_clock.cc:648] HybridClock initialized: now 1790422847266587 us; error 34 us; skew 500 ppm
Sep 26 11:40:47 dist-test-slave-xxxx krb5kdc[12574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422847, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:47.269274 13185 init.cc:377] Logged in from keytab as oryx/127.6.96.60@KRBTEST.COM (short username oryx)
I20260926 11:40:47.270201 13185 webserver.cc:460] Webserver started at http://127.6.96.60:33961/ using document root <none> and password file <none>
I20260926 11:40:47.270732 13185 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:47.270787 13185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:47.271035 13185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:47.273080 13185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/data/instance:
uuid: "51ff483a96c5450cb2b5cb4951b240a6"
format_stamp: "Formatted at 2026-09-26 11:40:47 on dist-test-slave-xxxx"
server_key: "e7e4641bb52ecc49b1d1ae49a32a50d7"
server_key_iv: "7dd57c95c252e974d801ba376a2deb7a"
server_key_version: "encryptionkey@0"
I20260926 11:40:47.273603 13185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/wal/instance:
uuid: "51ff483a96c5450cb2b5cb4951b240a6"
format_stamp: "Formatted at 2026-09-26 11:40:47 on dist-test-slave-xxxx"
server_key: "e7e4641bb52ecc49b1d1ae49a32a50d7"
server_key_iv: "7dd57c95c252e974d801ba376a2deb7a"
server_key_version: "encryptionkey@0"
I20260926 11:40:47.277405 13185 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 11:40:47.279873 13286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:47.281122 13185 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:40:47.281245 13185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/wal
uuid: "51ff483a96c5450cb2b5cb4951b240a6"
format_stamp: "Formatted at 2026-09-26 11:40:47 on dist-test-slave-xxxx"
server_key: "e7e4641bb52ecc49b1d1ae49a32a50d7"
server_key_iv: "7dd57c95c252e974d801ba376a2deb7a"
server_key_version: "encryptionkey@0"
I20260926 11:40:47.281344 13185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:47.295466 13185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:47.298583 13185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:47.298759 13185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:47.305254 13185 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.60:35405
I20260926 11:40:47.305507 13338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.60:35405 every 8 connection(s)
I20260926 11:40:47.306190 13185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/data/info.pb
I20260926 11:40:47.308893 13339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:47.310220 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 13185
I20260926 11:40:47.313237 13339 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 11:40:47 dist-test-slave-xxxx krb5kdc[12574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.96.60@KRBTEST.COM
Sep 26 11:40:47 dist-test-slave-xxxx krb5kdc[12574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422847, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.60@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM
I20260926 11:40:47.323490 12658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.315003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38240 (local address 127.6.96.62:44995)
0926 11:40:47.315119 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:47.315124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:47.315403 (+ 279us) server_negotiation.cc:407] Connection header received
0926 11:40:47.315522 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:47.315526 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:47.315571 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:47.315636 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:47.316260 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.316766 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.317656 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.317882 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.317967 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.320202 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:47.320206 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:47.320208 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:47.320236 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:47.321468 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.321993 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.321997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.321999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.322080 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.322373 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.322376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.322378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.322551 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:47.322643 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:47.323218 (+ 575us) server_negotiation.cc:299] Negotiation successful
0926 11:40:47.323370 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 11:40:47.324496 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.314901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.0.0.1:38240)
0926 11:40:47.315276 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:47.315322 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:47.315436 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:47.315690 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:47.315695 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:47.316030 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:47.316143 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.316148 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.316881 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.316884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.317545 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.317553 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.318016 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.318020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.318111 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.318575 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:47.318590 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:47.320023 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:47.321571 (+ 1548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.321578 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.321586 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.321901 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.322159 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.322162 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.322164 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.322292 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.322655 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:47.322660 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:47.322951 (+ 291us) client_negotiation.cc:769] Sending connection context
0926 11:40:47.323169 (+ 218us) client_negotiation.cc:240] Negotiation successful
0926 11:40:47.323473 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"spinlock_wait_cycles":2304,"thread_start_us":153,"threads_started":1}
I20260926 11:40:47.324664 13341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.311782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57992 (local address 127.6.96.60:35405)
0926 11:40:47.312242 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:47.312257 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:47.312294 (+ 37us) server_negotiation.cc:407] Connection header received
0926 11:40:47.312949 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:47.312966 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:47.313298 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:47.313626 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:47.314384 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.315782 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.316589 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.317049 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.317194 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.319639 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:47.319646 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:47.319659 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:47.319683 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:47.322514 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.322945 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.322950 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.322952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.323051 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.323300 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.323303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.323305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.323550 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:47.324251 (+ 701us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:47.324349 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 11:40:47.324476 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"spinlock_wait_cycles":1792,"thread_start_us":137,"threads_started":1}
I20260926 11:40:47.325615 13339 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 11:40:47 dist-test-slave-xxxx krb5kdc[12574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422847, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.60@KRBTEST.COM for oryx/127.6.96.61@KRBTEST.COM
I20260926 11:40:47.333376 13343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.326162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35012 (local address 127.6.96.61:39733)
0926 11:40:47.326440 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:47.326445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:47.326459 (+ 14us) server_negotiation.cc:407] Connection header received
0926 11:40:47.326497 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:47.326500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:47.326547 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:47.326629 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:47.327016 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.327699 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.328484 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.328753 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.328858 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.330721 (+ 1863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:47.330724 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:47.330726 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:47.330750 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:47.331934 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.332299 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.332303 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.332305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.332379 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.332610 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.332613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.332615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.332796 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:47.332876 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:47.333107 (+ 231us) server_negotiation.cc:299] Negotiation successful
0926 11:40:47.333223 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":92,"threads_started":1}
I20260926 11:40:47.333372 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.326044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:39733 (local address 127.0.0.1:35012)
0926 11:40:47.326178 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:47.326188 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:47.326243 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:47.326653 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:47.326656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:47.326792 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:47.326917 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.326923 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.327793 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.327796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.328328 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.328337 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.328869 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.328872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.328946 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.329350 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:47.329364 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:47.330585 (+ 1221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:47.332033 (+ 1448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.332036 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.332038 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.332214 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.332461 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.332464 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.332466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.332535 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.332880 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:47.332883 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:47.332968 (+ 85us) client_negotiation.cc:769] Sending connection context
0926 11:40:47.333050 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 11:40:47.333179 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260926 11:40:47.334220 13339 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.60" port: 35405 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 11:40:47 dist-test-slave-xxxx krb5kdc[12574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422847, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.60@KRBTEST.COM for oryx/127.6.96.60@KRBTEST.COM
I20260926 11:40:47.341441 13341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.334719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58002 (local address 127.6.96.60:35405)
0926 11:40:47.334821 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:47.334851 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:47.334863 (+ 12us) server_negotiation.cc:407] Connection header received
0926 11:40:47.334958 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:47.334960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:47.334995 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:47.335053 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:47.335391 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.336217 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.336887 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.337114 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.337193 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.338946 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:47.338950 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:47.338952 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:47.338973 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:47.340102 (+ 1129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.340460 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.340463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.340465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.340528 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.340733 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.340736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.340738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.340898 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:47.340973 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:47.341177 (+ 204us) server_negotiation.cc:299] Negotiation successful
0926 11:40:47.341302 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":50}
I20260926 11:40:47.341441 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.334679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:35405 (local address 127.0.0.1:58002)
0926 11:40:47.334772 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:47.334785 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:47.334875 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:47.335070 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:47.335072 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:47.335213 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:47.335300 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.335305 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.336312 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.336315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.336797 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.336803 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.337197 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.337201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.337294 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.337658 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:47.337668 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:47.338818 (+ 1150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:47.340189 (+ 1371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.340192 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.340195 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.340383 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.340596 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.340599 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.340601 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.340667 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.340973 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:47.340976 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:47.341052 (+ 76us) client_negotiation.cc:769] Sending connection context
0926 11:40:47.341130 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 11:40:47.341264 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":2}
I20260926 11:40:47.344332 13339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6: Bootstrap starting.
I20260926 11:40:47.346684 13339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:47.347415 13339 log.cc:824] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:47.349375 13339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6: No bootstrap required, opened a new log
I20260926 11:40:47.352447 13339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 35405 } }
I20260926 11:40:47.352660 13339 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:47.352699 13339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51ff483a96c5450cb2b5cb4951b240a6, State: Initialized, Role: FOLLOWER
I20260926 11:40:47.353139 13339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 35405 } }
I20260926 11:40:47.353624 13345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 35405 } } }
I20260926 11:40:47.353753 13345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:40:47.354323 13339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 11:40:47.358733 13356 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 11:40:47.358868 13356 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 11:40:47.362069 13185 master_runner.cc:428] Detected that this master 51ff483a96c5450cb2b5cb4951b240a6 is joining an existing cluster
I20260926 11:40:47.362134 13185 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.96.60:35405
I20260926 11:40:47.373741 13341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.364783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58018 (local address 127.6.96.60:35405)
0926 11:40:47.364912 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:47.364917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:47.367050 (+ 2133us) server_negotiation.cc:407] Connection header received
0926 11:40:47.367169 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:47.367172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:47.367208 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:47.367248 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:47.367694 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.368271 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.369288 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.369632 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.369696 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.370890 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:47.370894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:47.370897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:47.370918 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:47.372127 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.372591 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.372594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.372596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.372661 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.372906 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.372909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.372910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.373107 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:47.373202 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:47.373487 (+ 285us) server_negotiation.cc:299] Negotiation successful
0926 11:40:47.373585 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":59}
I20260926 11:40:47.373905 12658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.364688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38256 (local address 127.6.96.62:44995)
0926 11:40:47.364920 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:47.364925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:47.365238 (+ 313us) server_negotiation.cc:407] Connection header received
0926 11:40:47.365343 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:47.365346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:47.365395 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:47.365474 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:47.366048 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.366718 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.368035 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.368384 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.368496 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.369845 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:47.369849 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:47.369852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:47.369880 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:47.371876 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.372755 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.372759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.372760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.372848 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.373084 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.373087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.373088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.373281 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:47.373425 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:47.373669 (+ 244us) server_negotiation.cc:299] Negotiation successful
0926 11:40:47.373776 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157}
I20260926 11:40:47.373937 13363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.364561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.0.0.1:38256)
0926 11:40:47.365142 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:47.365157 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:47.365260 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:47.365487 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:47.365490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:47.365738 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:47.365874 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.365882 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.366850 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.366853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.367912 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.367923 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.368510 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.368513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.368594 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.369237 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:47.369251 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:47.369694 (+ 443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:47.372172 (+ 2478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.372175 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.372177 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.372506 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.372940 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.372942 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.372943 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.373005 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.373416 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:47.373419 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:47.373505 (+ 86us) client_negotiation.cc:769] Sending connection context
0926 11:40:47.373607 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 11:40:47.373739 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":86,"threads_started":1}
I20260926 11:40:47.374538 13362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.364991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:35405 (local address 127.0.0.1:58018)
0926 11:40:47.366983 (+ 1992us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:47.366996 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:47.367094 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:47.367335 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:47.367337 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:47.367509 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:47.367600 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.367605 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.368388 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.368390 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.369181 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.369191 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.369816 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.369819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.369907 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.370353 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:47.370367 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:47.370743 (+ 376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:47.372218 (+ 1475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.372220 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.372223 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.372514 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.372732 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.372734 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.372736 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.372816 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.373235 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:47.373238 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:47.373321 (+ 83us) client_negotiation.cc:769] Sending connection context
0926 11:40:47.374201 (+ 880us) client_negotiation.cc:240] Negotiation successful
0926 11:40:47.374368 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1930,"mutex_wait_us":3,"spinlock_wait_cycles":7936,"thread_start_us":84,"threads_started":1}
I20260926 11:40:47.377605 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 12681
Sep 26 11:40:47 dist-test-slave-xxxx krb5kdc[12574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422846, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.62@KRBTEST.COM for oryx/127.6.96.60@KRBTEST.COM
I20260926 11:40:47.385367 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:38253
--local_ip_for_outbound_sockets=127.6.96.1
--tserver_master_addrs=127.6.96.62:44995
--webserver_port=39091
--webserver_interface=127.6.96.1
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.1.keytab
--principal=oryx/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:42279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.96.62:44995,127.6.96.61:39733
--tserver_master_addrs=127.6.96.62:44995,127.6.96.61:39733,127.6.96.60:35405 with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 11:40:47.411403 13176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.378383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:35405 (local address 127.0.0.1:58024)
0926 11:40:47.378483 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:47.378496 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:47.378569 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:47.378948 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:47.378952 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:47.379146 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:47.379253 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.379260 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.380188 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.380191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.380725 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.380733 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.381131 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.381135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.381222 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.381618 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:47.381631 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:47.382794 (+ 1163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:47.409550 (+ 26756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.409554 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.409557 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.409830 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.410213 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.410216 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.410219 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.410317 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.410803 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:47.410806 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:47.410940 (+ 134us) client_negotiation.cc:769] Sending connection context
0926 11:40:47.411041 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 11:40:47.411222 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260926 11:40:47.411691 13343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.364800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35020 (local address 127.6.96.61:39733)
0926 11:40:47.365005 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:47.365009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:47.365355 (+ 346us) server_negotiation.cc:407] Connection header received
0926 11:40:47.365580 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:47.365583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:47.365635 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:47.365684 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:47.366399 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.367129 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.369762 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.370533 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.370612 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.408294 (+ 37682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:47.408299 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:47.408301 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:47.408340 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:47.409949 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.410410 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.410414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.410416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.410511 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.410797 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.410801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.410802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.411030 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:47.411093 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:47.411418 (+ 325us) server_negotiation.cc:299] Negotiation successful
0926 11:40:47.411541 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260926 11:40:47.412050 13341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.378457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58024 (local address 127.6.96.60:35405)
0926 11:40:47.378549 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:47.378553 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:47.378566 (+ 13us) server_negotiation.cc:407] Connection header received
0926 11:40:47.378687 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:47.378690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:47.378739 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:47.378807 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:47.379393 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.380087 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.380825 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.381042 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.381121 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.383316 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:47.383320 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:47.383323 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:47.383344 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:47.409391 (+ 26047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.409965 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.409968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.409970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.410054 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.410412 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.410415 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.410417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.410672 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:47.410755 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:47.411784 (+ 1029us) server_negotiation.cc:299] Negotiation successful
0926 11:40:47.411895 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 11:40:47.413025 13364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.364561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:39733 (local address 127.0.0.1:35020)
0926 11:40:47.365264 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:47.365277 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:47.365419 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:47.365846 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:47.365849 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:47.366119 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:47.366246 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.366254 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.368046 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.368049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.369649 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.369656 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.370989 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.370992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.371059 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.407513 (+ 36454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:47.407537 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:47.408058 (+ 521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:47.410066 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.410069 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.410071 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.410297 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.410617 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.410619 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.410621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.410706 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.411159 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:47.411162 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:47.411251 (+ 89us) client_negotiation.cc:769] Sending connection context
0926 11:40:47.412718 (+ 1467us) client_negotiation.cc:240] Negotiation successful
0926 11:40:47.412862 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"mutex_wait_us":422,"thread_start_us":112,"threads_started":1}
I20260926 11:40:47.413044 12619 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.96.60:35405: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6"
member_type: NON_VOTER
last_known_addr {
host: "127.6.96.60"
port: 35405
}
attrs {
promote: true
}
}
dest_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7"
cas_config_opid_index: 5
I20260926 11:40:47.413686 12619 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: NON_VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: true } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: NON_VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: true } }
I20260926 11:40:47.415046 13181 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 51ff483a96c5450cb2b5cb4951b240a6 (127.6.96.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: NON_VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: true } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: NON_VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: true } } }
I20260926 11:40:47.416600 13181 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.96.60:35405
I20260926 11:40:47.416783 13181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: NON_VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: true } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: NON_VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: true } } }
I20260926 11:40:47.416874 13181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:47.417569 13185 master.cc:537] Master@127.6.96.60:35405 shutting down...
I20260926 11:40:47.417712 13313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:47.419215 13313 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [term 1 FOLLOWER]: Refusing update from remote peer 10fbe49d79eb4051ab5b16c45aba3cc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 11:40:47.420114 13179 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: NON_VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 11:40:47.422103 12604 consensus_peers.cc:487] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 -> Peer 51ff483a96c5450cb2b5cb4951b240a6 (127.6.96.60:35405): Couldn't send request to peer 51ff483a96c5450cb2b5cb4951b240a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 11:40:47.423553 13185 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:47.423843 13185 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:47.423949 13185 tablet_replica.cc:330] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6: stopping tablet replica
I20260926 11:40:47.424255 13176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.414858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:39733 (local address 127.0.0.1:35032)
0926 11:40:47.415248 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:47.415262 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:47.415342 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:47.415719 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:47.415722 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:47.415738 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:47.415820 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.415825 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.416762 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.416765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.422652 (+ 5887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.422665 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.423808 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.423812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.423907 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.423958 (+ 51us) client_negotiation.cc:769] Sending connection context
0926 11:40:47.424033 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 11:40:47.424112 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323}
I20260926 11:40:47.424639 13343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.415060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35032 (local address 127.6.96.61:39733)
0926 11:40:47.415163 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:47.415167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:47.415415 (+ 248us) server_negotiation.cc:407] Connection header received
0926 11:40:47.415458 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:47.415460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:47.415515 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:47.415595 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:47.415920 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.416621 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.422803 (+ 6182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.423481 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.423588 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.423744 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:47.424447 (+ 703us) server_negotiation.cc:299] Negotiation successful
0926 11:40:47.424526 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 11:40:47.425117 13231 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [term 1 LEARNER]: Refusing update from remote peer 10fbe49d79eb4051ab5b16c45aba3cc7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 11:40:47.425544 13179 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: NON_VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260926 11:40:47.427321 13231 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 51ff483a96c5450cb2b5cb4951b240a6 (127.6.96.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: NON_VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: true } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: NON_VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: true } } }
I20260926 11:40:47.428880 13260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: NON_VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: true } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: NON_VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: true } } }
I20260926 11:40:47.429018 13260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [sys.catalog]: This master's current role is: LEARNER
I20260926 11:40:47.430403 13179 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7: attempting to promote NON_VOTER 65753213026347eab7c588f0e98074c3 to VOTER
I20260926 11:40:47.430905 13179 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: NON_VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: true } }
I20260926 11:40:47.428864 13367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10fbe49d79eb4051ab5b16c45aba3cc7. Latest consensus state: current_term: 1 leader_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: NON_VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: true } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: NON_VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: true } } }
I20260926 11:40:47.431563 13367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [sys.catalog]: This master's current role is: LEARNER
I20260926 11:40:47.432170 13231 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [term 1 LEARNER]: Refusing update from remote peer 10fbe49d79eb4051ab5b16c45aba3cc7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260926 11:40:47.432554 13176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.432207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:35405 (local address 127.0.0.1:58026)
0926 11:40:47.432407 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:47.432476 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:35405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260926 11:40:47.432524 13178 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260926 11:40:47.433432 13176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.433147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:35405 (local address 127.0.0.1:58032)
0926 11:40:47.433318 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:47.433365 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:35405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":44}
W20260926 11:40:47.433583 12604 consensus_peers.cc:487] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 -> Peer 51ff483a96c5450cb2b5cb4951b240a6 (127.6.96.60:35405): Couldn't send request to peer 51ff483a96c5450cb2b5cb4951b240a6. Status: Network error: Client connection negotiation failed: client connection to 127.6.96.60:35405: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 11:40:47.434295 13180 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 65753213026347eab7c588f0e98074c3 (127.6.96.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: NON_VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: true } } }
I20260926 11:40:47.435593 13180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: NON_VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: true } } }
I20260926 11:40:47.435706 13180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:47.435755 13181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: NON_VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: true } } }
I20260926 11:40:47.435829 13181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:47.436380 13231 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 65753213026347eab7c588f0e98074c3 (127.6.96.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: NON_VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: true } } }
I20260926 11:40:47.437661 13367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: NON_VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: true } } }
I20260926 11:40:47.437772 13367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:40:47.442454 13185 master.cc:552] Master@127.6.96.60:35405 shutdown complete.
I20260926 11:40:47.442500 13185 master_runner.cc:305] Clearing existing system tablet
I20260926 11:40:47.442700 13185 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:47.442744 13185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:47.443749 13185 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:47.444460 13369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:47.444671 13185 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:47.444792 13185 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:47.444836 13185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/wal
uuid: "51ff483a96c5450cb2b5cb4951b240a6"
format_stamp: "Formatted at 2026-09-26 11:40:47 on dist-test-slave-xxxx"
server_key: "e7e4641bb52ecc49b1d1ae49a32a50d7"
server_key_iv: "7dd57c95c252e974d801ba376a2deb7a"
server_key_version: "encryptionkey@0"
I20260926 11:40:47.446355 13185 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:47.448165 13185 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 11:40:47.448240 13185 log.cc:1196] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260926 11:40:47.448446 13185 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6: Deleting consensus metadata
I20260926 11:40:47.448606 13185 master_runner.cc:315] Copying system tablet from 127.6.96.62:44995
I20260926 11:40:47.449641 13185 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6: tablet copy: Beginning tablet copy session from remote peer at address 127.6.96.62:44995
I20260926 11:40:47.457306 12658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.450204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38266 (local address 127.6.96.62:44995)
0926 11:40:47.450342 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:47.450347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:47.450516 (+ 169us) server_negotiation.cc:407] Connection header received
0926 11:40:47.450631 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:47.450635 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:47.450679 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:47.450744 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:47.451184 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.451766 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.452567 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.452790 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.452883 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.454304 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:47.454308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:47.454311 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:47.454337 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:47.455766 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.456197 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.456201 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.456203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.456282 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.456524 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.456526 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.456528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.456703 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:47.456785 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:47.457031 (+ 246us) server_negotiation.cc:299] Negotiation successful
0926 11:40:47.457136 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 11:40:47.457310 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.450115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.0.0.1:38266)
0926 11:40:47.450427 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:47.450440 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:47.450553 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:47.450767 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:47.450769 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:47.450976 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:47.451075 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.451081 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.451874 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.451877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.452459 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.452467 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.452895 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.452899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.452987 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.453781 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:47.453792 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:47.454100 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:47.455889 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.455892 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.455894 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.456110 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.456366 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.456369 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.456371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.456449 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.456785 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:47.456788 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:47.456888 (+ 100us) client_negotiation.cc:769] Sending connection context
0926 11:40:47.456969 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 11:40:47.457110 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":91,"threads_started":1}
I20260926 11:40:47.458132 12639 tablet_copy_service.cc:140] P 10fbe49d79eb4051ab5b16c45aba3cc7: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 51ff483a96c5450cb2b5cb4951b240a6 ({username='oryx', principal='oryx/127.6.96.60@KRBTEST.COM'} at 127.0.0.1:38266)
I20260926 11:40:47.458217 12639 tablet_copy_service.cc:161] P 10fbe49d79eb4051ab5b16c45aba3cc7: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 51ff483a96c5450cb2b5cb4951b240a6 at {username='oryx', principal='oryx/127.6.96.60@KRBTEST.COM'} at 127.0.0.1:38266: session id = 51ff483a96c5450cb2b5cb4951b240a6-00000000000000000000000000000000
I20260926 11:40:47.459159 12639 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 11:40:47.460307 13185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 11:40:47.462141 13185 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6: tablet copy: Starting download of 0 data blocks...
I20260926 11:40:47.462334 13185 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6: tablet copy: Starting download of 1 WAL segments...
I20260926 11:40:47.463694 13185 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 11:40:47.465139 12639 tablet_copy_service.cc:342] P 10fbe49d79eb4051ab5b16c45aba3cc7: Request end of tablet copy session 51ff483a96c5450cb2b5cb4951b240a6-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.96.60@KRBTEST.COM'} at 127.0.0.1:38266
I20260926 11:40:47.465211 12639 tablet_copy_service.cc:433] P 10fbe49d79eb4051ab5b16c45aba3cc7: ending tablet copy session 51ff483a96c5450cb2b5cb4951b240a6-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 51ff483a96c5450cb2b5cb4951b240a6
I20260926 11:40:47.472117 13185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:47.473608 13379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:47.473774 13376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:47.473889 13185 server_base.cc:1034] running on GCE node
W20260926 11:40:47.474014 13377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:47.475055 13185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:47.475363 13185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:47.476514 13185 hybrid_clock.cc:648] HybridClock initialized: now 1790422847476504 us; error 30 us; skew 500 ppm
Sep 26 11:40:47 dist-test-slave-xxxx krb5kdc[12574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422847, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:47.477934 13185 init.cc:377] Logged in from keytab as oryx/127.6.96.60@KRBTEST.COM (short username oryx)
I20260926 11:40:47.478369 13185 webserver.cc:460] Webserver started at http://127.6.96.60:39199/ using document root <none> and password file <none>
I20260926 11:40:47.478546 13185 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:47.478581 13185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:47.479787 13185 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:47.480659 13386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:47.480940 13185 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:47.481009 13185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/wal
uuid: "51ff483a96c5450cb2b5cb4951b240a6"
format_stamp: "Formatted at 2026-09-26 11:40:47 on dist-test-slave-xxxx"
server_key: "e7e4641bb52ecc49b1d1ae49a32a50d7"
server_key_iv: "7dd57c95c252e974d801ba376a2deb7a"
server_key_version: "encryptionkey@0"
I20260926 11:40:47.481093 13185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:47.516525 13185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:47.518344 13185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:47.522765 13185 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.60:35405
I20260926 11:40:47.522789 13438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.60:35405 every 8 connection(s)
I20260926 11:40:47.523303 13185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/master-2/data/info.pb
I20260926 11:40:47.524634 13439 sys_catalog.cc:263] Verifying existing consensus state
I20260926 11:40:47.525451 13439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6: Bootstrap starting.
I20260926 11:40:47.534636 13439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 11:40:47.534937 13439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6: Bootstrap complete.
I20260926 11:40:47.535471 13439 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: NON_VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: true } }
I20260926 11:40:47.535590 13439 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 51ff483a96c5450cb2b5cb4951b240a6, State: Initialized, Role: LEARNER
I20260926 11:40:47.535691 13439 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: NON_VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: true } }
I20260926 11:40:47.536144 13439 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:40:47.536341 13445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: NON_VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: true } } }
I20260926 11:40:47.536551 13445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [sys.catalog]: This master's current role is: LEARNER
I20260926 11:40:47.538717 13456 catalog_manager.cc:1259] Loaded cluster ID: ab48db96286a4772873e443a7a2f1221
I20260926 11:40:47.538801 13456 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6: loading cluster ID for follower catalog manager: success
I20260926 11:40:47.540555 13456 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6: acquiring CA information for follower catalog manager: success
I20260926 11:40:47.541666 13456 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 11:40:47.542766 13365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:47.543102 13365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:47.543185 13365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:47.549523 13365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:47.549616 13365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:47.549705 13365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:40:47.557529 13365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:42279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.1.keytab
--principal=oryx/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:38253
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=39091
--tserver_master_addrs=127.6.96.62:44995,127.6.96.61:39733,127.6.96.60:35405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:47.558946 13365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:47.560130 13365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:47.569783 13458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:47.569927 13365 server_base.cc:1034] running on GCE node
W20260926 11:40:47.569689 13459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:47.569718 13461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:47.570542 13365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:47.571231 13365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:47.572392 13365 hybrid_clock.cc:648] HybridClock initialized: now 1790422847572366 us; error 34 us; skew 500 ppm
Sep 26 11:40:47 dist-test-slave-xxxx krb5kdc[12574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422847, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:47.575153 13365 init.cc:377] Logged in from keytab as oryx/127.6.96.1@KRBTEST.COM (short username oryx)
I20260926 11:40:47.576128 13365 webserver.cc:460] Webserver started at http://127.6.96.1:39091/ using document root <none> and password file <none>
I20260926 11:40:47.576719 13365 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:47.576798 13365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:47.581090 13365 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 11:40:47.583316 13468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:47.584767 13365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:40:47.584880 13365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "10f2ba5d863c469c9084ea7824990d8d"
format_stamp: "Formatted at 2026-09-26 11:40:46 on dist-test-slave-xxxx"
server_key: "cb864365836311b5f0ac80c35ec4ecf4"
server_key_iv: "ff075e36e9f1cee7255572e8f44e88c2"
server_key_version: "encryptionkey@0"
I20260926 11:40:47.585461 13365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:47.613292 13365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:47.616550 13365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:47.616725 13365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:47.617318 13365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:47.618184 13365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:47.618232 13365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:47.618271 13365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:47.618300 13365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:47.619431 12944 heartbeater.cc:503] Master 127.6.96.62:44995 was elected leader, sending a full tablet report...
I20260926 11:40:47.626621 13365 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:38253
I20260926 11:40:47.626649 13581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:38253 every 8 connection(s)
I20260926 11:40:47.627692 13365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:40:47.633060 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 13365
I20260926 11:40:47.633236 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 12815
Sep 26 11:40:47 dist-test-slave-xxxx krb5kdc[12574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422847, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM
Sep 26 11:40:47 dist-test-slave-xxxx krb5kdc[12574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422847, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.61@KRBTEST.COM
Sep 26 11:40:47 dist-test-slave-xxxx krb5kdc[12574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422847, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.60@KRBTEST.COM
I20260926 11:40:47.640391 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:46263
--local_ip_for_outbound_sockets=127.6.96.2
--tserver_master_addrs=127.6.96.62:44995
--webserver_port=42047
--webserver_interface=127.6.96.2
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.2.keytab
--principal=oryx/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:42279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.96.62:44995,127.6.96.61:39733
--tserver_master_addrs=127.6.96.62:44995,127.6.96.61:39733,127.6.96.60:35405 with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 11:40:47.644824 13343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.631189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:39011 (local address 127.6.96.61:39733)
0926 11:40:47.631305 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:47.631309 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:47.631371 (+ 62us) server_negotiation.cc:407] Connection header received
0926 11:40:47.631704 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:47.631708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:47.631766 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:47.631816 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:47.633282 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.633973 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.635792 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.636179 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.636258 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.640840 (+ 4582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:47.640844 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:47.640846 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:47.640876 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:47.643110 (+ 2234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.643539 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.643542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.643544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.643619 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.643880 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.643884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.643885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.644084 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:47.644145 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:47.644533 (+ 388us) server_negotiation.cc:299] Negotiation successful
0926 11:40:47.644658 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 11:40:47.645083 13586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.629477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:45499 (local address 127.6.96.60:35405)
0926 11:40:47.629753 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:47.629757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:47.630623 (+ 866us) server_negotiation.cc:407] Connection header received
0926 11:40:47.631763 (+ 1140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:47.631766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:47.631831 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:47.631906 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:47.633282 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.635615 (+ 2333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.636666 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.637408 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.637522 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.640729 (+ 3207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:47.640732 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:47.640735 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:47.640766 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:47.642345 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.643018 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.643022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.643023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.643108 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.643504 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.643507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.643508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.643752 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:47.643839 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:47.644782 (+ 943us) server_negotiation.cc:299] Negotiation successful
0926 11:40:47.644923 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":88,"threads_started":1}
I20260926 11:40:47.645975 13589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.631074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:39733 (local address 127.6.96.1:39011)
0926 11:40:47.631306 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:47.631318 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:47.631493 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:47.631975 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:47.631982 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:47.632489 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:47.633125 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.633144 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.634138 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.634141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.635675 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.635685 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.636500 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.636504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.636653 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.637349 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:47.637364 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:47.640671 (+ 3307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:47.643222 (+ 2551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.643225 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.643227 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.643434 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.643717 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.643720 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.643722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.643786 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.644207 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:47.644209 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:47.644276 (+ 67us) client_negotiation.cc:769] Sending connection context
0926 11:40:47.645039 (+ 763us) client_negotiation.cc:240] Negotiation successful
0926 11:40:47.645171 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":148,"spinlock_wait_cycles":4864,"thread_start_us":93,"threads_started":1}
I20260926 11:40:47.646222 13588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.629960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:35405 (local address 127.6.96.1:45499)
0926 11:40:47.630479 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:47.630520 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:47.631485 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:47.632007 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:47.632010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:47.632490 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:47.633125 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.633144 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.635745 (+ 2601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.635749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.636550 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.636557 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.637653 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.637657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.637751 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.638228 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:47.638243 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:47.640335 (+ 2092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:47.642482 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.642489 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.642511 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.642905 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.643199 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.643202 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.643204 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.643398 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.643889 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:47.643896 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:47.644257 (+ 361us) client_negotiation.cc:769] Sending connection context
0926 11:40:47.644467 (+ 210us) client_negotiation.cc:240] Negotiation successful
0926 11:40:47.644846 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":112,"spinlock_wait_cycles":63104,"thread_start_us":131,"threads_started":1}
I20260926 11:40:47.646801 12658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.630142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:39219 (local address 127.6.96.62:44995)
0926 11:40:47.630265 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:47.630270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:47.631115 (+ 845us) server_negotiation.cc:407] Connection header received
0926 11:40:47.631671 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:47.631674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:47.631728 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:47.631800 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:47.633287 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.634030 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.635813 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.636156 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.636248 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.640508 (+ 4260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:47.640512 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:47.640515 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:47.640545 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:47.642874 (+ 2329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.643342 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.643346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.643348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.643432 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.643698 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.643702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.643704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.643928 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:47.644008 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:47.646532 (+ 2524us) server_negotiation.cc:299] Negotiation successful
0926 11:40:47.646661 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 11:40:47.647353 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.629960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.6.96.1:39219)
0926 11:40:47.631025 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:47.631038 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:47.631483 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:47.631974 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:47.631981 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:47.632469 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:47.633134 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.633145 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.634322 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.634325 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.635681 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.635691 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.636435 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.636439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.636643 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.637299 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:47.637320 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:47.640311 (+ 2991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:47.643010 (+ 2699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.643014 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.643016 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.643236 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.643524 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.643527 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.643529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.643605 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.644047 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:47.644050 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:47.644255 (+ 205us) client_negotiation.cc:769] Sending connection context
0926 11:40:47.644468 (+ 213us) client_negotiation.cc:240] Negotiation successful
0926 11:40:47.647164 (+ 2696us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":978,"mutex_wait_us":183,"spinlock_wait_cycles":9728,"thread_start_us":156,"threads_started":1}
I20260926 11:40:47.647657 13582 heartbeater.cc:347] Connected to a master server at 127.6.96.60:35405
I20260926 11:40:47.647899 13582 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:47.648170 13583 heartbeater.cc:347] Connected to a master server at 127.6.96.62:44995
I20260926 11:40:47.648252 13583 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:47.648464 13583 heartbeater.cc:511] Master 127.6.96.62:44995 requested a full tablet report, sending...
I20260926 11:40:47.648488 13584 heartbeater.cc:347] Connected to a master server at 127.6.96.61:39733
I20260926 11:40:47.648465 13582 heartbeater.cc:511] Master 127.6.96.60:35405 requested a full tablet report, sending...
I20260926 11:40:47.648559 13584 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:47.648696 13584 heartbeater.cc:511] Master 127.6.96.61:39733 requested a full tablet report, sending...
I20260926 11:40:47.649822 12619 ts_manager.cc:194] Re-registered known tserver with Master: 10f2ba5d863c469c9084ea7824990d8d (127.6.96.1:38253)
I20260926 11:40:47.650075 13221 ts_manager.cc:194] Registered new tserver with Master: 10f2ba5d863c469c9084ea7824990d8d (127.6.96.1:38253)
I20260926 11:40:47.650077 13403 ts_manager.cc:194] Registered new tserver with Master: 10f2ba5d863c469c9084ea7824990d8d (127.6.96.1:38253)
I20260926 11:40:47.650581 12619 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.96.1@KRBTEST.COM'} at 127.6.96.1:39219
W20260926 11:40:47.755920 13590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:47.756246 13590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:47.756296 13590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:47.760409 13590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:47.760476 13590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:47.760536 13590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:40:47.765946 13590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:42279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.2.keytab
--principal=oryx/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:46263
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=42047
--tserver_master_addrs=127.6.96.62:44995,127.6.96.61:39733,127.6.96.60:35405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:47.767086 13590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:47.768151 13590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:47.778105 13596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:47.778098 13595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:47.778098 13598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:47.778154 13590 server_base.cc:1034] running on GCE node
I20260926 11:40:47.778915 13590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:47.779596 13590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:47.780771 13590 hybrid_clock.cc:648] HybridClock initialized: now 1790422847780749 us; error 36 us; skew 500 ppm
Sep 26 11:40:47 dist-test-slave-xxxx krb5kdc[12574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422847, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:47.783692 13590 init.cc:377] Logged in from keytab as oryx/127.6.96.2@KRBTEST.COM (short username oryx)
I20260926 11:40:47.784704 13590 webserver.cc:460] Webserver started at http://127.6.96.2:42047/ using document root <none> and password file <none>
I20260926 11:40:47.785282 13590 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:47.785365 13590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:47.789521 13590 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260926 11:40:47.791767 13605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:47.793303 13590 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:40:47.793426 13590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "ec00c3f1615d44b6adeedbe52d77f35c"
format_stamp: "Formatted at 2026-09-26 11:40:46 on dist-test-slave-xxxx"
server_key: "e6f81715ce6570948c03ca8694239223"
server_key_iv: "4d58726f50de4a0264382f3403251f9b"
server_key_version: "encryptionkey@0"
I20260926 11:40:47.794147 13590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:47.809368 13590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:47.812446 13590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:47.812645 13590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:47.812652 13078 heartbeater.cc:503] Master 127.6.96.62:44995 was elected leader, sending a full tablet report...
I20260926 11:40:47.813339 13590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:47.814275 13590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:47.814322 13590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:47.814360 13590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:47.814375 13590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:47.822680 13590 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:46263
I20260926 11:40:47.822701 13718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:46263 every 8 connection(s)
I20260926 11:40:47.823688 13590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:40:47.827243 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 13590
I20260926 11:40:47.827383 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 12949
I20260926 11:40:47.833874 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:37879
--local_ip_for_outbound_sockets=127.6.96.3
--tserver_master_addrs=127.6.96.62:44995
--webserver_port=44513
--webserver_interface=127.6.96.3
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.3.keytab
--principal=oryx/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:42279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.96.62:44995,127.6.96.61:39733
--tserver_master_addrs=127.6.96.62:44995,127.6.96.61:39733,127.6.96.60:35405 with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 11:40:47 dist-test-slave-xxxx krb5kdc[12574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422847, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM
Sep 26 11:40:47 dist-test-slave-xxxx krb5kdc[12574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422847, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.61@KRBTEST.COM
Sep 26 11:40:47 dist-test-slave-xxxx krb5kdc[12574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422847, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.60@KRBTEST.COM
I20260926 11:40:47.841514 13586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.825445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:34831 (local address 127.6.96.60:35405)
0926 11:40:47.825580 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:47.825584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:47.826749 (+ 1165us) server_negotiation.cc:407] Connection header received
0926 11:40:47.828594 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:47.828597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:47.828651 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:47.828699 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:47.830896 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.831670 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.832837 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.833160 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.833229 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.837206 (+ 3977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:47.837210 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:47.837213 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:47.837251 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:47.838979 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.839680 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.839684 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.839686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.839771 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.840186 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.840190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.840192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.840403 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:47.840469 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:47.841228 (+ 759us) server_negotiation.cc:299] Negotiation successful
0926 11:40:47.841353 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 11:40:47.841514 12658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.825388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:41947 (local address 127.6.96.62:44995)
0926 11:40:47.825508 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:47.825512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:47.827063 (+ 1551us) server_negotiation.cc:407] Connection header received
0926 11:40:47.828360 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:47.828363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:47.828436 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:47.828505 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:47.830430 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.831251 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.832832 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.833139 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.833206 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.837207 (+ 4001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:47.837210 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:47.837213 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:47.837241 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:47.839072 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.839680 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.839684 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.839685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.839770 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.840187 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.840190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.840192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.840403 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:47.840469 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:47.841241 (+ 772us) server_negotiation.cc:299] Negotiation successful
0926 11:40:47.841352 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 11:40:47.841554 13343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.825812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:58613 (local address 127.6.96.61:39733)
0926 11:40:47.825923 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:47.825927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:47.826553 (+ 626us) server_negotiation.cc:407] Connection header received
0926 11:40:47.828640 (+ 2087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:47.828643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:47.828691 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:47.828751 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:47.831245 (+ 2494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.831958 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.832964 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.833277 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.833343 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.837202 (+ 3859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:47.837205 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:47.837208 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:47.837234 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:47.839650 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.840204 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.840207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.840209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.840296 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:47.840603 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:47.840606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:47.840608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:47.840833 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:47.840913 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:47.841265 (+ 352us) server_negotiation.cc:299] Negotiation successful
0926 11:40:47.841396 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 11:40:47.843163 13724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.825911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:35405 (local address 127.6.96.2:34831)
0926 11:40:47.826676 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:47.826687 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:47.828490 (+ 1803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:47.829033 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:47.829043 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:47.830655 (+ 1612us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:47.830752 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.830757 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.831790 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.831793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.832704 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.832714 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.833401 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.833405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.833667 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.834756 (+ 1089us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:47.834770 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:47.836981 (+ 2211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:47.839104 (+ 2123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.839110 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.839127 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.839568 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.839878 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.839881 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.839883 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.840082 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.840548 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:47.840555 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:47.840950 (+ 395us) client_negotiation.cc:769] Sending connection context
0926 11:40:47.841699 (+ 749us) client_negotiation.cc:240] Negotiation successful
0926 11:40:47.842117 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"mutex_wait_us":227,"spinlock_wait_cycles":37760,"thread_start_us":90,"threads_started":1}
I20260926 11:40:47.843415 13725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.825911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.6.96.2:41947)
0926 11:40:47.826955 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:47.826967 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:47.828128 (+ 1161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:47.829225 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:47.829227 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:47.829543 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:47.830276 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.830297 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.831413 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.831416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.832697 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.832708 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.833400 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.833404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.833674 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.834232 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:47.834251 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:47.837073 (+ 2822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:47.839167 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.839170 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.839172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.839568 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.839878 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.839881 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.839883 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.840086 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.840548 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:47.840555 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:47.840950 (+ 395us) client_negotiation.cc:769] Sending connection context
0926 11:40:47.841663 (+ 713us) client_negotiation.cc:240] Negotiation successful
0926 11:40:47.842056 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":955,"mutex_wait_us":283,"spinlock_wait_cycles":64896,"thread_start_us":86,"threads_started":1}
I20260926 11:40:47.843168 13723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.825911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:39733 (local address 127.6.96.2:58613)
0926 11:40:47.826405 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:47.826456 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:47.828421 (+ 1965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:47.829033 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:47.829043 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:47.830981 (+ 1938us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:47.831093 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.831099 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.832079 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.832081 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.832861 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.832868 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.833482 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.833485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.833711 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.834372 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:47.834388 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:47.836981 (+ 2593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:47.839786 (+ 2805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.839789 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.839791 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.840086 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.840417 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:47.840420 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:47.840422 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:47.840503 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:47.840973 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:47.840976 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:47.841079 (+ 103us) client_negotiation.cc:769] Sending connection context
0926 11:40:47.842375 (+ 1296us) client_negotiation.cc:240] Negotiation successful
0926 11:40:47.842516 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":92,"spinlock_wait_cycles":5632,"thread_start_us":133,"threads_started":1}
I20260926 11:40:47.844429 13720 heartbeater.cc:347] Connected to a master server at 127.6.96.62:44995
I20260926 11:40:47.844661 13720 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:47.845108 13720 heartbeater.cc:511] Master 127.6.96.62:44995 requested a full tablet report, sending...
I20260926 11:40:47.845520 13719 heartbeater.cc:347] Connected to a master server at 127.6.96.60:35405
I20260926 11:40:47.845594 13719 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:47.845758 13719 heartbeater.cc:511] Master 127.6.96.60:35405 requested a full tablet report, sending...
I20260926 11:40:47.844432 13721 heartbeater.cc:347] Connected to a master server at 127.6.96.61:39733
I20260926 11:40:47.845957 13721 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:47.846082 13721 heartbeater.cc:511] Master 127.6.96.61:39733 requested a full tablet report, sending...
I20260926 11:40:47.846114 12619 ts_manager.cc:194] Re-registered known tserver with Master: ec00c3f1615d44b6adeedbe52d77f35c (127.6.96.2:46263)
I20260926 11:40:47.846710 13403 ts_manager.cc:194] Registered new tserver with Master: ec00c3f1615d44b6adeedbe52d77f35c (127.6.96.2:46263)
I20260926 11:40:47.846740 13221 ts_manager.cc:194] Registered new tserver with Master: ec00c3f1615d44b6adeedbe52d77f35c (127.6.96.2:46263)
I20260926 11:40:47.846799 12619 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.96.2@KRBTEST.COM'} at 127.6.96.2:41947
W20260926 11:40:47.946542 13726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:47.946863 13726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:47.946930 13726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:47.951028 13726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:47.951084 13726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:47.951159 13726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:40:47.956389 13726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:42279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.3.keytab
--principal=oryx/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:37879
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=44513
--tserver_master_addrs=127.6.96.62:44995,127.6.96.61:39733,127.6.96.60:35405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:47.957427 13726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:47.958425 13726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:47.967583 13732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:47.967612 13734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:47.967612 13731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:47.967940 13726 server_base.cc:1034] running on GCE node
I20260926 11:40:47.968305 13726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:47.968931 13726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:47.970089 13726 hybrid_clock.cc:648] HybridClock initialized: now 1790422847970063 us; error 35 us; skew 500 ppm
Sep 26 11:40:47 dist-test-slave-xxxx krb5kdc[12574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422847, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:47.977002 13726 init.cc:377] Logged in from keytab as oryx/127.6.96.3@KRBTEST.COM (short username oryx)
I20260926 11:40:47.977877 13726 webserver.cc:460] Webserver started at http://127.6.96.3:44513/ using document root <none> and password file <none>
I20260926 11:40:47.978390 13726 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:47.978466 13726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:47.982380 13726 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260926 11:40:47.984400 13741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:47.985667 13726 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 11:40:47.985759 13726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "c8733268bb584adbb5deeaac640b5e3f"
format_stamp: "Formatted at 2026-09-26 11:40:46 on dist-test-slave-xxxx"
server_key: "8521467f1534b0c488d9001c04a44477"
server_key_iv: "8b326471796eeefdbfcc4cd03a15d999"
server_key_version: "encryptionkey@0"
I20260926 11:40:47.986260 13726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:47.993930 13742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.990390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:35405 (local address 127.0.0.1:58040)
0926 11:40:47.990604 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:47.990616 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:47.990742 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:47.991169 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:47.991172 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:47.991189 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:47.991266 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.991270 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.992018 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.992021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.992784 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:47.992793 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.993511 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.993516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:47.993614 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.993637 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 11:40:47.993704 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 11:40:47.993792 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":66,"threads_started":1}
I20260926 11:40:47.993924 13586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:47.990452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58040 (local address 127.6.96.60:35405)
0926 11:40:47.990570 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:47.990574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:47.990694 (+ 120us) server_negotiation.cc:407] Connection header received
0926 11:40:47.990930 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:47.990934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:47.990997 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:47.991052 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:47.991402 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.991909 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.992917 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:47.993343 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:47.993436 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:47.993579 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:47.993732 (+ 153us) server_negotiation.cc:299] Negotiation successful
0926 11:40:47.993799 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 11:40:47.994495 13413 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260926 11:40:47.994860 13445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10fbe49d79eb4051ab5b16c45aba3cc7. Latest consensus state: current_term: 1 leader_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: NON_VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: true } } }
I20260926 11:40:47.995000 13445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [sys.catalog]: This master's current role is: LEARNER
I20260926 11:40:48.006122 13726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:48.008908 13726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:48.009053 13726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:48.009593 13726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:48.010394 13726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:48.010447 13726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:48.010485 13726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:48.010499 13726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:48.018622 13726 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:37879
I20260926 11:40:48.018648 13855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:37879 every 8 connection(s)
I20260926 11:40:48.019640 13726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:40:48.020481 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 13726
Sep 26 11:40:48 dist-test-slave-xxxx krb5kdc[12574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422847, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM
Sep 26 11:40:48 dist-test-slave-xxxx krb5kdc[12574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422847, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.60@KRBTEST.COM
Sep 26 11:40:48 dist-test-slave-xxxx krb5kdc[12574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422847, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.61@KRBTEST.COM
I20260926 11:40:48.035769 13870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:48.023243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35048 (local address 127.6.96.61:39733)
0926 11:40:48.024019 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:48.024023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:48.024034 (+ 11us) server_negotiation.cc:407] Connection header received
0926 11:40:48.024074 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:48.024077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:48.024122 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:48.025109 (+ 987us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:48.025212 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.026067 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.028530 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.028851 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.029789 (+ 938us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:48.031342 (+ 1553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:48.031346 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:48.031349 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:48.031377 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:48.033226 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:48.034056 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:48.034060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:48.034062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:48.034144 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:48.034479 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:48.034483 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:48.034484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:48.034713 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:48.035393 (+ 680us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:48.035474 (+ 81us) server_negotiation.cc:299] Negotiation successful
0926 11:40:48.035585 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706,"thread_start_us":60,"threads_started":1}
I20260926 11:40:48.038693 13343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:48.022442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:48861 (local address 127.6.96.61:39733)
0926 11:40:48.022558 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:48.022561 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:48.023000 (+ 439us) server_negotiation.cc:407] Connection header received
0926 11:40:48.025166 (+ 2166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:48.025169 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:48.025206 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:48.025252 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:48.026968 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.027690 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.028787 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.029113 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.029195 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:48.034950 (+ 5755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:48.034953 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:48.034956 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:48.034981 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:48.036844 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:48.037266 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:48.037269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:48.037271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:48.037348 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:48.037644 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:48.037647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:48.037649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:48.037852 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:48.037925 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:48.038422 (+ 497us) server_negotiation.cc:299] Negotiation successful
0926 11:40:48.038543 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 11:40:48.038693 12658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:48.022158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:54775 (local address 127.6.96.62:44995)
0926 11:40:48.022357 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:48.022362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:48.023559 (+ 1197us) server_negotiation.cc:407] Connection header received
0926 11:40:48.024508 (+ 949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:48.024511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:48.024558 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:48.024629 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:48.026213 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.026960 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.028672 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.028991 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.029078 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:48.032847 (+ 3769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:48.032850 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:48.032852 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:48.032876 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:48.035668 (+ 2792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:48.036624 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:48.036628 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:48.036630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:48.036716 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:48.037178 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:48.037181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:48.037183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:48.037394 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:48.037479 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:48.038402 (+ 923us) server_negotiation.cc:299] Negotiation successful
0926 11:40:48.038523 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260926 11:40:48.038884 13586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:48.021641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:55021 (local address 127.6.96.60:35405)
0926 11:40:48.022045 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:48.022049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:48.023930 (+ 1881us) server_negotiation.cc:407] Connection header received
0926 11:40:48.024403 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:48.024406 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:48.024457 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:48.024507 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:48.026770 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.027546 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.028774 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.029052 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.029116 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:48.033932 (+ 4816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:48.033937 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:48.033940 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:48.033968 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:48.036396 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:48.037450 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:48.037454 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:48.037456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:48.037557 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:48.038001 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:48.038005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:48.038007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:48.038210 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:48.038282 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:48.038605 (+ 323us) server_negotiation.cc:299] Negotiation successful
0926 11:40:48.038719 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306}
I20260926 11:40:48.040376 13871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:48.023389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38268 (local address 127.6.96.62:44995)
0926 11:40:48.024063 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:48.024066 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:48.026136 (+ 2070us) server_negotiation.cc:407] Connection header received
0926 11:40:48.026298 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:48.026311 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:48.026360 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:48.026420 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:48.030293 (+ 3873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.031071 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.032225 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.032540 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.032616 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:48.034509 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:48.034513 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:48.034516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:48.034545 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:48.037177 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:48.038002 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:48.038006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:48.038007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:48.038093 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:48.038962 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:48.038966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:48.038968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:48.039165 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:48.039276 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:48.040098 (+ 822us) server_negotiation.cc:299] Negotiation successful
0926 11:40:48.040237 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":51,"threads_started":1}
I20260926 11:40:48.040498 13864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:48.022009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:39733 (local address 127.6.96.3:48861)
0926 11:40:48.022808 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:48.022892 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:48.025062 (+ 2170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:48.025356 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:48.025359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:48.025535 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:48.026848 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:48.026854 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.027839 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.027842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:48.028686 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:48.028692 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.029348 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.029351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:48.029870 (+ 519us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:48.032460 (+ 2590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:48.032475 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:48.034773 (+ 2298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:48.036942 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:48.036945 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:48.036947 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:48.037159 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:48.037453 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:48.037456 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:48.037458 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:48.037541 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:48.037978 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:48.037981 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:48.038122 (+ 141us) client_negotiation.cc:769] Sending connection context
0926 11:40:48.038922 (+ 800us) client_negotiation.cc:240] Negotiation successful
0926 11:40:48.039413 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"mutex_wait_us":1401,"spinlock_wait_cycles":10368,"thread_start_us":131,"threads_started":1}
I20260926 11:40:48.040926 13868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:48.022913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:35405 (local address 127.6.96.3:55021)
0926 11:40:48.023838 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:48.023849 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:48.024179 (+ 330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:48.024862 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:48.024871 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:48.026549 (+ 1678us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:48.026661 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:48.026667 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.027698 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.027701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:48.028611 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:48.028619 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.029452 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.029456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:48.029573 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:48.030729 (+ 1156us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:48.030746 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:48.033390 (+ 2644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:48.037094 (+ 3704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:48.037097 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:48.037099 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:48.037298 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:48.037693 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:48.037696 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:48.037698 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:48.037828 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:48.038347 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:48.038350 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:48.038438 (+ 88us) client_negotiation.cc:769] Sending connection context
0926 11:40:48.039229 (+ 791us) client_negotiation.cc:240] Negotiation successful
0926 11:40:48.039379 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":874,"mutex_wait_us":691,"thread_start_us":83,"threads_started":1}
I20260926 11:40:48.041729 13858 heartbeater.cc:347] Connected to a master server at 127.6.96.61:39733
I20260926 11:40:48.041967 13858 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:48.042059 13869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:48.023194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58054 (local address 127.6.96.60:35405)
0926 11:40:48.024170 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:48.024182 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:48.024193 (+ 11us) server_negotiation.cc:407] Connection header received
0926 11:40:48.024231 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:48.024233 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:48.024276 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:48.025196 (+ 920us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:48.025317 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.026058 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.028034 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.028398 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.029948 (+ 1550us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:48.032060 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:48.032064 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:48.032066 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:48.032092 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:48.037885 (+ 5793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:48.038742 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:48.038745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:48.038746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:48.038855 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:48.039766 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:48.039770 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:48.039771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:48.041028 (+ 1257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:48.041709 (+ 681us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:48.041794 (+ 85us) server_negotiation.cc:299] Negotiation successful
0926 11:40:48.041909 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":918,"thread_start_us":75,"threads_started":1}
I20260926 11:40:48.042223 13856 heartbeater.cc:347] Connected to a master server at 127.6.96.60:35405
I20260926 11:40:48.042296 13856 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:48.040768 13865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:48.022427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.6.96.3:54775)
0926 11:40:48.023459 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:48.023471 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:48.024190 (+ 719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:48.024862 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:48.024871 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:48.025386 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:48.026068 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:48.026084 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.027097 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.027101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:48.028542 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:48.028552 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.029305 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.029311 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:48.029559 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:48.030206 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:48.030229 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:48.032678 (+ 2449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:48.035961 (+ 3283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:48.035969 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:48.035991 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:48.036455 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:48.036853 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:48.036856 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:48.036858 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:48.037063 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:48.037534 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:48.037542 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:48.038123 (+ 581us) client_negotiation.cc:769] Sending connection context
0926 11:40:48.038934 (+ 811us) client_negotiation.cc:240] Negotiation successful
0926 11:40:48.039537 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":975,"mutex_wait_us":10,"spinlock_wait_cycles":6272,"thread_start_us":107,"threads_started":1}
I20260926 11:40:48.042550 13856 heartbeater.cc:511] Master 127.6.96.60:35405 requested a full tablet report, sending...
I20260926 11:40:48.042798 13857 heartbeater.cc:347] Connected to a master server at 127.6.96.62:44995
I20260926 11:40:48.042886 13857 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:48.043017 13857 heartbeater.cc:511] Master 127.6.96.62:44995 requested a full tablet report, sending...
I20260926 11:40:48.043601 12619 ts_manager.cc:194] Re-registered known tserver with Master: c8733268bb584adbb5deeaac640b5e3f (127.6.96.3:37879)
I20260926 11:40:48.043656 13403 ts_manager.cc:194] Registered new tserver with Master: c8733268bb584adbb5deeaac640b5e3f (127.6.96.3:37879)
I20260926 11:40:48.042550 13858 heartbeater.cc:511] Master 127.6.96.61:39733 requested a full tablet report, sending...
I20260926 11:40:48.044298 12619 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.96.3@KRBTEST.COM'} at 127.6.96.3:54775
I20260926 11:40:48.044344 13221 ts_manager.cc:194] Registered new tserver with Master: c8733268bb584adbb5deeaac640b5e3f (127.6.96.3:37879)
I20260926 11:40:48.044081 12618 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38268:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 11:40:48.047628 12618 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:40:48.050066 13875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: NON_VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: true } } }
I20260926 11:40:48.050226 13875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:48.053241 13875 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7: attempting to promote NON_VOTER 51ff483a96c5450cb2b5cb4951b240a6 to VOTER
I20260926 11:40:48.053550 13875 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: false } }
I20260926 11:40:48.055106 13231 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [term 1 FOLLOWER]: Refusing update from remote peer 10fbe49d79eb4051ab5b16c45aba3cc7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260926 11:40:48.055359 13412 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [term 1 LEARNER]: Refusing update from remote peer 10fbe49d79eb4051ab5b16c45aba3cc7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260926 11:40:48.055774 13876 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260926 11:40:48.056037 13876 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260926 11:40:48.057464 13874 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 51ff483a96c5450cb2b5cb4951b240a6 (127.6.96.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: false } } }
I20260926 11:40:48.057847 13412 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 51ff483a96c5450cb2b5cb4951b240a6 (127.6.96.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: false } } }
I20260926 11:40:48.057987 13231 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 51ff483a96c5450cb2b5cb4951b240a6 (127.6.96.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: false } } }
I20260926 11:40:48.058920 13874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: false } } }
I20260926 11:40:48.059020 13874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:48.059157 13874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: false } } }
I20260926 11:40:48.059242 13874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:48.059217 13876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: false } } }
I20260926 11:40:48.059314 13876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:48.059608 13445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: false } } }
I20260926 11:40:48.060168 13445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:40:48.060446 13884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: false } } }
I20260926 11:40:48.060740 13884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:40:48.067227 13742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:48.062340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:38253 (local address 127.0.0.1:60342)
0926 11:40:48.062440 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:48.062455 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:48.062529 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:48.063729 (+ 1200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:48.063732 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:48.063749 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:48.063845 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:48.063851 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.065031 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.065034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:48.065835 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:48.065843 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.066817 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.066821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:48.066925 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:48.066950 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 11:40:48.067027 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 11:40:48.067097 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260926 11:40:48.067283 13888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:48.062951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60342 (local address 127.6.96.1:38253)
0926 11:40:48.063251 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:48.063260 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:48.063279 (+ 19us) server_negotiation.cc:407] Connection header received
0926 11:40:48.063355 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:48.063360 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:48.063583 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:48.063995 (+ 412us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:48.064125 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.064932 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.065977 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.066669 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.066773 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:48.066957 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:48.067060 (+ 103us) server_negotiation.cc:299] Negotiation successful
0926 11:40:48.067148 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":97,"threads_started":1}
I20260926 11:40:48.067821 13887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:48.062546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:46263 (local address 127.0.0.1:49662)
0926 11:40:48.063027 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:48.063038 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:48.063116 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:48.063927 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:48.063929 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:48.063944 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:48.064033 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:48.064039 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.065277 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.065280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:48.066445 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:48.066453 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.067454 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.067457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:48.067539 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:48.067558 (+ 19us) client_negotiation.cc:769] Sending connection context
0926 11:40:48.067631 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 11:40:48.067693 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":110,"threads_started":1}
I20260926 11:40:48.067880 13886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:48.062698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49662 (local address 127.6.96.2:46263)
0926 11:40:48.063504 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:48.063516 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:48.063540 (+ 24us) server_negotiation.cc:407] Connection header received
0926 11:40:48.063615 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:48.063621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:48.063812 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:48.063950 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:48.064178 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.065157 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.066559 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.067332 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.067433 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:48.067564 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:48.067660 (+ 96us) server_negotiation.cc:299] Negotiation successful
0926 11:40:48.067746 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712,"thread_start_us":500,"threads_started":1}
I20260926 11:40:48.070428 13516 tablet_service.cc:1467] Processing CreateTablet for tablet 3db8d79758f24637b0da60308c2f98e4 (DEFAULT_TABLE table=test-table [id=1dc9b9d263f94426a6b9d3adf93070f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:48.070428 13653 tablet_service.cc:1467] Processing CreateTablet for tablet 3db8d79758f24637b0da60308c2f98e4 (DEFAULT_TABLE table=test-table [id=1dc9b9d263f94426a6b9d3adf93070f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:48.071405 13516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3db8d79758f24637b0da60308c2f98e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:48.071435 13653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3db8d79758f24637b0da60308c2f98e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:48.072124 13889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:48.063188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57136 (local address 127.6.96.3:37879)
0926 11:40:48.063482 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:48.063492 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:48.063606 (+ 114us) server_negotiation.cc:407] Connection header received
0926 11:40:48.063709 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:48.063714 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:48.063894 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:48.064018 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:48.064347 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.066173 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.068311 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.071504 (+ 3193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.071630 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:48.071766 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:48.071881 (+ 115us) server_negotiation.cc:299] Negotiation successful
0926 11:40:48.071975 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":101,"threads_started":1}
I20260926 11:40:48.072633 13885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:48.062815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:37879 (local address 127.0.0.1:57136)
0926 11:40:48.063510 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:48.063523 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:48.063613 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:48.064137 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:48.064139 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:48.064155 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:48.064227 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:48.064233 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.066287 (+ 2054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.066290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:48.067812 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:48.067821 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.071632 (+ 3811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.071636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:48.071731 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:48.071756 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 11:40:48.071834 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 11:40:48.071912 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":78,"threads_started":1}
I20260926 11:40:48.076056 13790 tablet_service.cc:1467] Processing CreateTablet for tablet 3db8d79758f24637b0da60308c2f98e4 (DEFAULT_TABLE table=test-table [id=1dc9b9d263f94426a6b9d3adf93070f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:48.077157 13790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3db8d79758f24637b0da60308c2f98e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:48.078614 13890 tablet_bootstrap.cc:492] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c: Bootstrap starting.
I20260926 11:40:48.080741 13890 tablet_bootstrap.cc:654] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:48.081471 13890 log.cc:824] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:48.083544 13890 tablet_bootstrap.cc:492] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c: No bootstrap required, opened a new log
I20260926 11:40:48.083760 13890 ts_tablet_manager.cc:1397] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 11:40:48.084239 13893 tablet_bootstrap.cc:492] T 3db8d79758f24637b0da60308c2f98e4 P c8733268bb584adbb5deeaac640b5e3f: Bootstrap starting.
I20260926 11:40:48.084863 13891 tablet_bootstrap.cc:492] T 3db8d79758f24637b0da60308c2f98e4 P 10f2ba5d863c469c9084ea7824990d8d: Bootstrap starting.
I20260926 11:40:48.086537 13893 tablet_bootstrap.cc:654] T 3db8d79758f24637b0da60308c2f98e4 P c8733268bb584adbb5deeaac640b5e3f: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:48.086891 13891 tablet_bootstrap.cc:654] T 3db8d79758f24637b0da60308c2f98e4 P 10f2ba5d863c469c9084ea7824990d8d: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:48.086903 13890 raft_consensus.cc:348] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f2ba5d863c469c9084ea7824990d8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38253 } } peers { permanent_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46263 } } peers { permanent_uuid: "c8733268bb584adbb5deeaac640b5e3f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37879 } }
I20260926 11:40:48.087153 13890 raft_consensus.cc:374] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:48.087203 13890 raft_consensus.cc:729] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec00c3f1615d44b6adeedbe52d77f35c, State: Initialized, Role: FOLLOWER
I20260926 11:40:48.087440 13893 log.cc:824] T 3db8d79758f24637b0da60308c2f98e4 P c8733268bb584adbb5deeaac640b5e3f: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:48.087599 13891 log.cc:824] T 3db8d79758f24637b0da60308c2f98e4 P 10f2ba5d863c469c9084ea7824990d8d: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:48.087652 13890 consensus_queue.cc:260] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f2ba5d863c469c9084ea7824990d8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38253 } } peers { permanent_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46263 } } peers { permanent_uuid: "c8733268bb584adbb5deeaac640b5e3f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37879 } }
I20260926 11:40:48.088567 13890 ts_tablet_manager.cc:1428] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260926 11:40:48.088660 13720 heartbeater.cc:503] Master 127.6.96.62:44995 was elected leader, sending a full tablet report...
I20260926 11:40:48.089658 13891 tablet_bootstrap.cc:492] T 3db8d79758f24637b0da60308c2f98e4 P 10f2ba5d863c469c9084ea7824990d8d: No bootstrap required, opened a new log
I20260926 11:40:48.089888 13891 ts_tablet_manager.cc:1397] T 3db8d79758f24637b0da60308c2f98e4 P 10f2ba5d863c469c9084ea7824990d8d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.005s
I20260926 11:40:48.094024 13891 raft_consensus.cc:348] T 3db8d79758f24637b0da60308c2f98e4 P 10f2ba5d863c469c9084ea7824990d8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f2ba5d863c469c9084ea7824990d8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38253 } } peers { permanent_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46263 } } peers { permanent_uuid: "c8733268bb584adbb5deeaac640b5e3f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37879 } }
I20260926 11:40:48.094235 13891 raft_consensus.cc:374] T 3db8d79758f24637b0da60308c2f98e4 P 10f2ba5d863c469c9084ea7824990d8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:48.094282 13891 raft_consensus.cc:729] T 3db8d79758f24637b0da60308c2f98e4 P 10f2ba5d863c469c9084ea7824990d8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10f2ba5d863c469c9084ea7824990d8d, State: Initialized, Role: FOLLOWER
I20260926 11:40:48.094689 13891 consensus_queue.cc:260] T 3db8d79758f24637b0da60308c2f98e4 P 10f2ba5d863c469c9084ea7824990d8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f2ba5d863c469c9084ea7824990d8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38253 } } peers { permanent_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46263 } } peers { permanent_uuid: "c8733268bb584adbb5deeaac640b5e3f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37879 } }
I20260926 11:40:48.095363 13583 heartbeater.cc:503] Master 127.6.96.62:44995 was elected leader, sending a full tablet report...
I20260926 11:40:48.095613 13891 ts_tablet_manager.cc:1428] T 3db8d79758f24637b0da60308c2f98e4 P 10f2ba5d863c469c9084ea7824990d8d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 11:40:48.096791 13893 tablet_bootstrap.cc:492] T 3db8d79758f24637b0da60308c2f98e4 P c8733268bb584adbb5deeaac640b5e3f: No bootstrap required, opened a new log
I20260926 11:40:48.097050 13893 ts_tablet_manager.cc:1397] T 3db8d79758f24637b0da60308c2f98e4 P c8733268bb584adbb5deeaac640b5e3f: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20260926 11:40:48.100924 13893 raft_consensus.cc:348] T 3db8d79758f24637b0da60308c2f98e4 P c8733268bb584adbb5deeaac640b5e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f2ba5d863c469c9084ea7824990d8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38253 } } peers { permanent_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46263 } } peers { permanent_uuid: "c8733268bb584adbb5deeaac640b5e3f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37879 } }
I20260926 11:40:48.101140 13893 raft_consensus.cc:374] T 3db8d79758f24637b0da60308c2f98e4 P c8733268bb584adbb5deeaac640b5e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:48.101186 13893 raft_consensus.cc:729] T 3db8d79758f24637b0da60308c2f98e4 P c8733268bb584adbb5deeaac640b5e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8733268bb584adbb5deeaac640b5e3f, State: Initialized, Role: FOLLOWER
I20260926 11:40:48.101552 13893 consensus_queue.cc:260] T 3db8d79758f24637b0da60308c2f98e4 P c8733268bb584adbb5deeaac640b5e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f2ba5d863c469c9084ea7824990d8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38253 } } peers { permanent_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46263 } } peers { permanent_uuid: "c8733268bb584adbb5deeaac640b5e3f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37879 } }
I20260926 11:40:48.102181 13857 heartbeater.cc:503] Master 127.6.96.62:44995 was elected leader, sending a full tablet report...
I20260926 11:40:48.104261 13893 ts_tablet_manager.cc:1428] T 3db8d79758f24637b0da60308c2f98e4 P c8733268bb584adbb5deeaac640b5e3f: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
W20260926 11:40:48.129673 13585 tablet.cc:2369] T 3db8d79758f24637b0da60308c2f98e4 P 10f2ba5d863c469c9084ea7824990d8d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 11:40:48.272606 13859 tablet.cc:2369] T 3db8d79758f24637b0da60308c2f98e4 P c8733268bb584adbb5deeaac640b5e3f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:40:48.298575 13896 raft_consensus.cc:482] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:40:48.298780 13896 raft_consensus.cc:504] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f2ba5d863c469c9084ea7824990d8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38253 } } peers { permanent_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46263 } } peers { permanent_uuid: "c8733268bb584adbb5deeaac640b5e3f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37879 } }
I20260926 11:40:48.299923 13896 leader_election.cc:288] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10f2ba5d863c469c9084ea7824990d8d (127.6.96.1:38253), c8733268bb584adbb5deeaac640b5e3f (127.6.96.3:37879)
I20260926 11:40:48.303624 13723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:48.300143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:37879 (local address 127.6.96.2:57517)
0926 11:40:48.300304 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:48.300317 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:48.300405 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:48.300664 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:48.300667 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:48.300682 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:48.300775 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:48.300780 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.301589 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.301592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:48.302482 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:48.302491 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.303233 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.303238 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:48.303337 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:48.303363 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 11:40:48.303434 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 11:40:48.303504 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":24,"spinlock_wait_cycles":11776}
I20260926 11:40:48.303651 13889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:48.300353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:57517 (local address 127.6.96.3:37879)
0926 11:40:48.300476 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:48.300480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:48.300493 (+ 13us) server_negotiation.cc:407] Connection header received
0926 11:40:48.300536 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:48.300538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:48.300574 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:48.300640 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:48.300867 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.301491 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.302600 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.303120 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.303199 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:48.303251 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:48.303458 (+ 207us) server_negotiation.cc:299] Negotiation successful
0926 11:40:48.303530 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 11:40:48.303789 13724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:48.300116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:38253 (local address 127.6.96.2:57327)
0926 11:40:48.300288 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:48.300302 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:48.300404 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:48.300639 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:48.300642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:48.300659 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:48.301095 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:48.301104 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.301862 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.301864 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:48.302702 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:48.302711 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.303415 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.303418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:48.303521 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:48.303542 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 11:40:48.303614 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 11:40:48.303680 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":19,"spinlock_wait_cycles":9344}
I20260926 11:40:48.303817 13888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:48.300214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:57327 (local address 127.6.96.1:38253)
0926 11:40:48.300326 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:48.300330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:48.300376 (+ 46us) server_negotiation.cc:407] Connection header received
0926 11:40:48.300508 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:48.300512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:48.300558 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:48.300612 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:48.301202 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.301759 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.302857 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.303308 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.303411 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:48.303467 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:48.303642 (+ 175us) server_negotiation.cc:299] Negotiation successful
0926 11:40:48.303712 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 11:40:48.304247 13810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3db8d79758f24637b0da60308c2f98e4" candidate_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8733268bb584adbb5deeaac640b5e3f" is_pre_election: true
I20260926 11:40:48.304394 13536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3db8d79758f24637b0da60308c2f98e4" candidate_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10f2ba5d863c469c9084ea7824990d8d" is_pre_election: true
I20260926 11:40:48.304549 13810 raft_consensus.cc:2454] T 3db8d79758f24637b0da60308c2f98e4 P c8733268bb584adbb5deeaac640b5e3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec00c3f1615d44b6adeedbe52d77f35c in term 0.
I20260926 11:40:48.304672 13536 raft_consensus.cc:2454] T 3db8d79758f24637b0da60308c2f98e4 P 10f2ba5d863c469c9084ea7824990d8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec00c3f1615d44b6adeedbe52d77f35c in term 0.
I20260926 11:40:48.305007 13606 leader_election.cc:302] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8733268bb584adbb5deeaac640b5e3f, ec00c3f1615d44b6adeedbe52d77f35c; no voters:
I20260926 11:40:48.305300 13896 raft_consensus.cc:2781] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:40:48.305375 13896 raft_consensus.cc:482] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:40:48.305410 13896 raft_consensus.cc:3037] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:48.306490 13896 raft_consensus.cc:504] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f2ba5d863c469c9084ea7824990d8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38253 } } peers { permanent_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46263 } } peers { permanent_uuid: "c8733268bb584adbb5deeaac640b5e3f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37879 } }
I20260926 11:40:48.306910 13896 leader_election.cc:288] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c [CANDIDATE]: Term 1 election: Requested vote from peers 10f2ba5d863c469c9084ea7824990d8d (127.6.96.1:38253), c8733268bb584adbb5deeaac640b5e3f (127.6.96.3:37879)
I20260926 11:40:48.307215 13536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3db8d79758f24637b0da60308c2f98e4" candidate_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10f2ba5d863c469c9084ea7824990d8d"
I20260926 11:40:48.307284 13810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3db8d79758f24637b0da60308c2f98e4" candidate_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8733268bb584adbb5deeaac640b5e3f"
I20260926 11:40:48.307337 13536 raft_consensus.cc:3037] T 3db8d79758f24637b0da60308c2f98e4 P 10f2ba5d863c469c9084ea7824990d8d [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:48.307407 13810 raft_consensus.cc:3037] T 3db8d79758f24637b0da60308c2f98e4 P c8733268bb584adbb5deeaac640b5e3f [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:48.308296 13536 raft_consensus.cc:2454] T 3db8d79758f24637b0da60308c2f98e4 P 10f2ba5d863c469c9084ea7824990d8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec00c3f1615d44b6adeedbe52d77f35c in term 1.
I20260926 11:40:48.308419 13810 raft_consensus.cc:2454] T 3db8d79758f24637b0da60308c2f98e4 P c8733268bb584adbb5deeaac640b5e3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec00c3f1615d44b6adeedbe52d77f35c in term 1.
I20260926 11:40:48.308560 13609 leader_election.cc:302] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10f2ba5d863c469c9084ea7824990d8d, ec00c3f1615d44b6adeedbe52d77f35c; no voters:
I20260926 11:40:48.308763 13896 raft_consensus.cc:2781] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:48.308930 13896 raft_consensus.cc:686] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c [term 1 LEADER]: Becoming Leader. State: Replica: ec00c3f1615d44b6adeedbe52d77f35c, State: Running, Role: LEADER
I20260926 11:40:48.309253 13896 consensus_queue.cc:237] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f2ba5d863c469c9084ea7824990d8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38253 } } peers { permanent_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46263 } } peers { permanent_uuid: "c8733268bb584adbb5deeaac640b5e3f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37879 } }
I20260926 11:40:48.312067 12618 catalog_manager.cc:5495] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c reported cstate change: term changed from 0 to 1, leader changed from <none> to ec00c3f1615d44b6adeedbe52d77f35c (127.6.96.2). New cstate: current_term: 1 leader_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f2ba5d863c469c9084ea7824990d8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8733268bb584adbb5deeaac640b5e3f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37879 } health_report { overall_health: UNKNOWN } } }
W20260926 11:40:48.325652 13722 tablet.cc:2369] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:40:48.359577 13886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:48.356249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49676 (local address 127.6.96.2:46263)
0926 11:40:48.356402 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:48.356407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:48.356442 (+ 35us) server_negotiation.cc:407] Connection header received
0926 11:40:48.356519 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:48.356521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:48.356576 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:48.356661 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:40:48.356892 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.357374 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.358181 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.358386 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.358477 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:48.358754 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:40:48.359215 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:40:48.359322 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:48.359380 (+ 58us) server_negotiation.cc:299] Negotiation successful
0926 11:40:48.359444 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 11:40:48.365365 13536 raft_consensus.cc:1261] T 3db8d79758f24637b0da60308c2f98e4 P 10f2ba5d863c469c9084ea7824990d8d [term 1 FOLLOWER]: Refusing update from remote peer ec00c3f1615d44b6adeedbe52d77f35c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:48.365365 13810 raft_consensus.cc:1261] T 3db8d79758f24637b0da60308c2f98e4 P c8733268bb584adbb5deeaac640b5e3f [term 1 FOLLOWER]: Refusing update from remote peer ec00c3f1615d44b6adeedbe52d77f35c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:48.366071 13899 consensus_queue.cc:1035] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8733268bb584adbb5deeaac640b5e3f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:48.366230 13896 consensus_queue.cc:1035] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c [LEADER]: Connected to new peer: Peer: permanent_uuid: "10f2ba5d863c469c9084ea7824990d8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:48.383379 12619 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:48.383561 12619 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:48.384793 13903 mvcc.cc:205] Tried to move back new op lower bound from 7333571986897477632 to 7333571986677280768. Current Snapshot: MvccSnapshot[applied={T|T < 7333571986897477632}]
I20260926 11:40:48.392666 12619 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 10fbe49d79eb4051ab5b16c45aba3cc7: Sending DeleteTablet for 3 replicas of tablet 3db8d79758f24637b0da60308c2f98e4
I20260926 11:40:48.393612 13653 tablet_service.cc:1514] Processing DeleteTablet for tablet 3db8d79758f24637b0da60308c2f98e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:48 UTC) from {username='oryx', principal='oryx/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:49662
I20260926 11:40:48.393606 13516 tablet_service.cc:1514] Processing DeleteTablet for tablet 3db8d79758f24637b0da60308c2f98e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:48 UTC) from {username='oryx', principal='oryx/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:60342
I20260926 11:40:48.394116 13912 tablet_replica.cc:330] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c: stopping tablet replica
I20260926 11:40:48.394340 13911 tablet_replica.cc:330] T 3db8d79758f24637b0da60308c2f98e4 P 10f2ba5d863c469c9084ea7824990d8d: stopping tablet replica
I20260926 11:40:48.395198 13790 tablet_service.cc:1514] Processing DeleteTablet for tablet 3db8d79758f24637b0da60308c2f98e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:48 UTC) from {username='oryx', principal='oryx/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:57136
I20260926 11:40:48.395315 13911 raft_consensus.cc:2229] T 3db8d79758f24637b0da60308c2f98e4 P 10f2ba5d863c469c9084ea7824990d8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:48.395751 13911 raft_consensus.cc:2258] T 3db8d79758f24637b0da60308c2f98e4 P 10f2ba5d863c469c9084ea7824990d8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:48.394815 13912 raft_consensus.cc:2229] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c [term 1 LEADER]: Raft consensus shutting down.
I20260926 11:40:48.396481 13912 raft_consensus.cc:2258] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:48.397845 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 12587
I20260926 11:40:48.399500 13912 ts_tablet_manager.cc:1905] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:48.402137 13912 ts_tablet_manager.cc:1918] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:40:48.402235 13912 log.cc:1196] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/wal/wals/3db8d79758f24637b0da60308c2f98e4
I20260926 11:40:48.402518 13912 ts_tablet_manager.cc:1939] T 3db8d79758f24637b0da60308c2f98e4 P ec00c3f1615d44b6adeedbe52d77f35c: Deleting consensus metadata
I20260926 11:40:48.403904 13911 ts_tablet_manager.cc:1905] T 3db8d79758f24637b0da60308c2f98e4 P 10f2ba5d863c469c9084ea7824990d8d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 11:40:48.404747 13606 connection.cc:535] server connection from 127.0.0.1:49662 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 11:40:48.406692 13913 tablet_replica.cc:330] T 3db8d79758f24637b0da60308c2f98e4 P c8733268bb584adbb5deeaac640b5e3f: stopping tablet replica
I20260926 11:40:48.406747 13911 ts_tablet_manager.cc:1918] T 3db8d79758f24637b0da60308c2f98e4 P 10f2ba5d863c469c9084ea7824990d8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:40:48.406879 13911 log.cc:1196] T 3db8d79758f24637b0da60308c2f98e4 P 10f2ba5d863c469c9084ea7824990d8d: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-0/wal/wals/3db8d79758f24637b0da60308c2f98e4
I20260926 11:40:48.407213 13911 ts_tablet_manager.cc:1939] T 3db8d79758f24637b0da60308c2f98e4 P 10f2ba5d863c469c9084ea7824990d8d: Deleting consensus metadata
W20260926 11:40:48.408284 13470 connection.cc:441] server connection from 127.0.0.1:60342 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:60342 (request call id 1) could send its response
I20260926 11:40:48.408469 13913 raft_consensus.cc:2229] T 3db8d79758f24637b0da60308c2f98e4 P c8733268bb584adbb5deeaac640b5e3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:48.408806 13913 raft_consensus.cc:2258] T 3db8d79758f24637b0da60308c2f98e4 P c8733268bb584adbb5deeaac640b5e3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:48.410182 13913 ts_tablet_manager.cc:1905] T 3db8d79758f24637b0da60308c2f98e4 P c8733268bb584adbb5deeaac640b5e3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:48.412885 13913 ts_tablet_manager.cc:1918] T 3db8d79758f24637b0da60308c2f98e4 P c8733268bb584adbb5deeaac640b5e3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:40:48.412972 13913 log.cc:1196] T 3db8d79758f24637b0da60308c2f98e4 P c8733268bb584adbb5deeaac640b5e3f: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/3db8d79758f24637b0da60308c2f98e4
I20260926 11:40:48.413098 13343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:48.396298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35056 (local address 127.6.96.61:39733)
0926 11:40:48.396406 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:48.396410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:48.403150 (+ 6740us) server_negotiation.cc:407] Connection header received
0926 11:40:48.403336 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:48.403339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:48.403387 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:48.403436 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:48.404518 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.405332 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.406968 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.407310 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.407385 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:48.409575 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:48.409578 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:48.409581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:48.409609 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:48.411195 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:48.411735 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:48.411738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:48.411740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:48.411828 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:48.412132 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:48.412135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:48.412136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:48.412358 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:48.412461 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:48.412831 (+ 370us) server_negotiation.cc:299] Negotiation successful
0926 11:40:48.412943 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 11:40:48.413260 13913 ts_tablet_manager.cc:1939] T 3db8d79758f24637b0da60308c2f98e4 P c8733268bb584adbb5deeaac640b5e3f: Deleting consensus metadata
W20260926 11:40:48.414342 13746 connection.cc:441] server connection from 127.0.0.1:57136 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57136 (request call id 1) could send its response
I20260926 11:40:48.420607 13586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:48.410112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58068 (local address 127.6.96.60:35405)
0926 11:40:48.410242 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:48.410246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:48.410989 (+ 743us) server_negotiation.cc:407] Connection header received
0926 11:40:48.411156 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:48.411159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:48.411206 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:48.411257 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:48.412499 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.413305 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.414549 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.414923 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.414996 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:48.416618 (+ 1622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:48.416622 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:48.416625 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:48.416651 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:48.418231 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:48.418678 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:48.418682 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:48.418683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:48.418764 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:48.419109 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:48.419113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:48.419114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:48.419327 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:48.419454 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:48.420027 (+ 573us) server_negotiation.cc:299] Negotiation successful
0926 11:40:48.420449 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 11:40:48.422343 13869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:48.396668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58066 (local address 127.6.96.60:35405)
0926 11:40:48.396798 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:48.396802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:48.414477 (+ 17675us) server_negotiation.cc:407] Connection header received
0926 11:40:48.414636 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:48.414639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:48.414682 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:48.414725 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:48.415461 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.416033 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.416860 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.417414 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.417476 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:48.418765 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:48.418769 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:48.418772 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:48.418792 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:48.420343 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:48.421095 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:48.421098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:48.421100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:48.421186 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:48.421453 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:48.421456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:48.421458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:48.421643 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:48.421737 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:48.422072 (+ 335us) server_negotiation.cc:299] Negotiation successful
0926 11:40:48.422191 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 11:40:48.427119 13343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:48.419161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35058 (local address 127.6.96.61:39733)
0926 11:40:48.419288 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:48.419293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:48.419697 (+ 404us) server_negotiation.cc:407] Connection header received
0926 11:40:48.419742 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:48.419745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:48.419795 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:48.419871 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:48.420653 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.421391 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.422542 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:48.422797 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:48.423056 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:48.424374 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:48.424377 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:48.424380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:48.424405 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:48.425691 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:48.426050 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:48.426054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:48.426055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:48.426126 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:48.426364 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:48.426367 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:48.426368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:48.426530 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:48.426636 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:48.426894 (+ 258us) server_negotiation.cc:299] Negotiation successful
0926 11:40:48.427004 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 11:40:49.396085 13921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.395679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.6.96.1:42769)
0926 11:40:49.395920 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:49.395996 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":83,"threads_started":1}
W20260926 11:40:49.396379 13583 heartbeater.cc:644] Failed to heartbeat to 127.6.96.62:44995 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
I20260926 11:40:49.396684 13922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.396316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.6.96.3:55697)
0926 11:40:49.396529 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:49.396595 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":61,"threads_started":1}
W20260926 11:40:49.396919 13857 heartbeater.cc:644] Failed to heartbeat to 127.6.96.62:44995 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
I20260926 11:40:49.398013 13923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.397667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.6.96.2:47871)
0926 11:40:49.397860 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:49.397940 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":56,"threads_started":1}
W20260926 11:40:49.398227 13720 heartbeater.cc:644] Failed to heartbeat to 127.6.96.62:44995 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
I20260926 11:40:49.890969 13924 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 10fbe49d79eb4051ab5b16c45aba3cc7)
I20260926 11:40:49.891181 13924 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: false } }
I20260926 11:40:49.892146 13924 leader_election.cc:288] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 10fbe49d79eb4051ab5b16c45aba3cc7 (127.6.96.62:44995), 65753213026347eab7c588f0e98074c3 (127.6.96.61:39733)
I20260926 11:40:49.892778 13925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.892309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.0.0.1:38764)
0926 11:40:49.892590 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:49.892673 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":84,"threads_started":1}
W20260926 11:40:49.892982 13390 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
I20260926 11:40:49.893829 13925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.893567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.0.0.1:38768)
0926 11:40:49.893712 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:49.893758 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260926 11:40:49.893947 13390 leader_election.cc:334] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 10fbe49d79eb4051ab5b16c45aba3cc7 (127.6.96.62:44995): Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
I20260926 11:40:49.896067 13929 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 10fbe49d79eb4051ab5b16c45aba3cc7)
I20260926 11:40:49.896263 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.892662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:39733 (local address 127.0.0.1:35068)
0926 11:40:49.892852 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:49.892866 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:49.892993 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:49.893255 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:49.893257 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:49.893278 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:49.893380 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:49.893386 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:49.894269 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:49.894271 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:49.895248 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:49.895258 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:49.895938 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:49.895941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:49.896010 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:49.896031 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 11:40:49.896084 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 11:40:49.896139 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":63,"threads_started":1}
I20260926 11:40:49.896237 13929 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: false } }
I20260926 11:40:49.896562 13926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.892460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35068 (local address 127.6.96.61:39733)
0926 11:40:49.892750 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:49.892755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:49.892955 (+ 200us) server_negotiation.cc:407] Connection header received
0926 11:40:49.893089 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:49.893092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:49.893142 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:49.893243 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:49.893487 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:49.894176 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:49.895356 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:49.895829 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:49.895942 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:49.896008 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:49.896119 (+ 111us) server_negotiation.cc:299] Negotiation successful
0926 11:40:49.896197 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":100,"threads_started":1}
I20260926 11:40:49.896960 13231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "51ff483a96c5450cb2b5cb4951b240a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "65753213026347eab7c588f0e98074c3" is_pre_election: true
I20260926 11:40:49.897162 13231 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51ff483a96c5450cb2b5cb4951b240a6 in term 1.
I20260926 11:40:49.897365 13929 leader_election.cc:288] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 10fbe49d79eb4051ab5b16c45aba3cc7 (127.6.96.62:44995), 51ff483a96c5450cb2b5cb4951b240a6 (127.6.96.60:35405)
I20260926 11:40:49.897585 13387 leader_election.cc:302] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51ff483a96c5450cb2b5cb4951b240a6, 65753213026347eab7c588f0e98074c3; no voters: 10fbe49d79eb4051ab5b16c45aba3cc7
I20260926 11:40:49.897809 13930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.897459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.0.0.1:38776)
0926 11:40:49.897667 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:49.897734 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":63,"threads_started":1}
I20260926 11:40:49.897946 13924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260926 11:40:49.897964 13208 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
I20260926 11:40:49.898011 13924 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 10fbe49d79eb4051ab5b16c45aba3cc7)
I20260926 11:40:49.898058 13924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [term 1 FOLLOWER]: Advancing to term 2
I20260926 11:40:49.899402 13924 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: false } }
I20260926 11:40:49.899745 13924 leader_election.cc:288] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [CANDIDATE]: Term 2 election: Requested vote from peers 10fbe49d79eb4051ab5b16c45aba3cc7 (127.6.96.62:44995), 65753213026347eab7c588f0e98074c3 (127.6.96.61:39733)
I20260926 11:40:49.900092 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.899840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.0.0.1:38790)
0926 11:40:49.899984 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:49.900031 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260926 11:40:49.900058 13931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.898523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.0.0.1:38782)
0926 11:40:49.899935 (+ 1412us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:49.899986 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1359}
I20260926 11:40:49.900046 13231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "51ff483a96c5450cb2b5cb4951b240a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "65753213026347eab7c588f0e98074c3"
I20260926 11:40:49.900171 13231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [term 1 FOLLOWER]: Advancing to term 2
I20260926 11:40:49.900671 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.900449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.0.0.1:38802)
0926 11:40:49.900567 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:49.900615 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260926 11:40:49.900732 13390 leader_election.cc:334] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 10fbe49d79eb4051ab5b16c45aba3cc7 (127.6.96.62:44995): Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
I20260926 11:40:49.901239 13231 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51ff483a96c5450cb2b5cb4951b240a6 in term 2.
W20260926 11:40:49.901465 13208 leader_election.cc:334] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 10fbe49d79eb4051ab5b16c45aba3cc7 (127.6.96.62:44995): Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
I20260926 11:40:49.901523 13387 leader_election.cc:302] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51ff483a96c5450cb2b5cb4951b240a6, 65753213026347eab7c588f0e98074c3; no voters: 10fbe49d79eb4051ab5b16c45aba3cc7
I20260926 11:40:49.901695 13924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 11:40:49.901880 13924 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [term 2 LEADER]: Becoming Leader. State: Replica: 51ff483a96c5450cb2b5cb4951b240a6, State: Running, Role: LEADER
I20260926 11:40:49.901968 13930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.897485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:35405 (local address 127.0.0.1:58078)
0926 11:40:49.898375 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:49.898388 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:49.898469 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:49.898849 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:49.898852 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:49.898873 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:49.898954 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:49.898959 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:49.900049 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:49.900053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:49.900934 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:49.900943 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:49.901609 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:49.901612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:49.901700 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:49.901727 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 11:40:49.901793 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 11:40:49.901857 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":842,"thread_start_us":69,"threads_started":1}
I20260926 11:40:49.902134 13924 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: false } }
I20260926 11:40:49.902360 13932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.897808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58078 (local address 127.6.96.60:35405)
0926 11:40:49.898642 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:49.898649 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:49.898660 (+ 11us) server_negotiation.cc:407] Connection header received
0926 11:40:49.898693 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:49.898696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:49.898734 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:49.898808 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:49.899052 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:49.899760 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:49.901054 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:49.901483 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:49.902119 (+ 636us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:49.902174 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:49.902202 (+ 28us) server_negotiation.cc:299] Negotiation successful
0926 11:40:49.902251 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":777,"thread_start_us":85,"threads_started":1}
I20260926 11:40:49.902688 13412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "65753213026347eab7c588f0e98074c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "51ff483a96c5450cb2b5cb4951b240a6" is_pre_election: true
I20260926 11:40:49.903173 13206 leader_election.cc:302] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 65753213026347eab7c588f0e98074c3; no voters: 10fbe49d79eb4051ab5b16c45aba3cc7, 51ff483a96c5450cb2b5cb4951b240a6
I20260926 11:40:49.903347 13929 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 11:40:49.903481 13934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51ff483a96c5450cb2b5cb4951b240a6. Latest consensus state: current_term: 2 leader_uuid: "51ff483a96c5450cb2b5cb4951b240a6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: false } } }
I20260926 11:40:49.903580 13934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:49.903815 13936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:40:49.905977 13936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:40:49.906348 13936 catalog_manager.cc:1259] Loaded cluster ID: ab48db96286a4772873e443a7a2f1221
I20260926 11:40:49.906395 13936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:40:49.907004 13936 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:40:49.907394 13936 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6: Loaded TSK: 0
I20260926 11:40:49.908183 13936 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:40:49.932139 13403 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58068:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 11:40:49.935226 13403 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:40:49.938360 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.938111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.0.0.1:38822)
0926 11:40:49.938223 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:49.938287 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260926 11:40:49.938431 13231 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [term 2 FOLLOWER]: Refusing update from remote peer 51ff483a96c5450cb2b5cb4951b240a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260926 11:40:49.939072 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.938857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.0.0.1:38836)
0926 11:40:49.938949 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:49.939000 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260926 11:40:49.938992 13934 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260926 11:40:49.939181 13390 consensus_peers.cc:487] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 -> Peer 10fbe49d79eb4051ab5b16c45aba3cc7 (127.6.96.62:44995): Couldn't send request to peer 10fbe49d79eb4051ab5b16c45aba3cc7. Status: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 11:40:49.940379 13929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51ff483a96c5450cb2b5cb4951b240a6. Latest consensus state: current_term: 2 leader_uuid: "51ff483a96c5450cb2b5cb4951b240a6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: false } } }
I20260926 11:40:49.940514 13929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:40:49.941427 13934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "51ff483a96c5450cb2b5cb4951b240a6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: false } } }
I20260926 11:40:49.941543 13934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:49.942936 13929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "51ff483a96c5450cb2b5cb4951b240a6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "10fbe49d79eb4051ab5b16c45aba3cc7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44995 } } peers { permanent_uuid: "65753213026347eab7c588f0e98074c3" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39733 } attrs { promote: false } } peers { permanent_uuid: "51ff483a96c5450cb2b5cb4951b240a6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 35405 } attrs { promote: false } } }
I20260926 11:40:49.943051 13929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65753213026347eab7c588f0e98074c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:40:49.951958 13945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.948021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:46263 (local address 127.0.0.1:49690)
0926 11:40:49.948459 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:49.948471 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:49.948553 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:49.948992 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:49.948995 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:49.949012 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:49.949112 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:49.949121 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:49.949839 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:49.949842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:49.950794 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:49.950803 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:49.951580 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:49.951583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:49.951648 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:49.951669 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 11:40:49.951744 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 11:40:49.951812 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":88,"threads_started":1}
I20260926 11:40:49.951982 13946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.948551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49690 (local address 127.6.96.2:46263)
0926 11:40:49.948772 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:49.948776 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:49.948789 (+ 13us) server_negotiation.cc:407] Connection header received
0926 11:40:49.948847 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:49.948850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:49.948892 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:49.948981 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:49.949234 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:49.949736 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:49.950924 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:49.951457 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:49.951563 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:49.951627 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:49.951787 (+ 160us) server_negotiation.cc:299] Negotiation successful
0926 11:40:49.951872 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":66,"threads_started":1}
I20260926 11:40:49.952231 13925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.947040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:38253 (local address 127.0.0.1:60348)
0926 11:40:49.947478 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:49.947491 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:49.947591 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:49.948143 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:49.948147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:49.948167 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:49.948272 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:49.948278 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:49.950160 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:49.950162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:49.951059 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:49.951068 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:49.951886 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:49.951889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:49.951981 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:49.952004 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 11:40:49.952047 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 11:40:49.952113 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376}
I20260926 11:40:49.952644 13943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.947142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60348 (local address 127.6.96.1:38253)
0926 11:40:49.947417 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:49.947422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:49.947556 (+ 134us) server_negotiation.cc:407] Connection header received
0926 11:40:49.947982 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:49.947985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:49.948038 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:49.948125 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:49.948396 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:49.949149 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:49.951166 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:49.951762 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:49.952347 (+ 585us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:49.952414 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:49.952446 (+ 32us) server_negotiation.cc:299] Negotiation successful
0926 11:40:49.952517 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":103,"threads_started":1}
I20260926 11:40:49.952733 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.947386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:37879 (local address 127.0.0.1:57138)
0926 11:40:49.947716 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:49.947727 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:49.947776 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:49.948262 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:49.948264 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:49.948281 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:49.948382 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:49.948388 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:49.949264 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:49.949267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:49.950047 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:49.950057 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:49.952426 (+ 2369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:49.952430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:49.952513 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:49.952533 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 11:40:49.952577 (+ 44us) client_negotiation.cc:240] Negotiation successful
0926 11:40:49.952639 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276}
I20260926 11:40:49.953052 13653 tablet_service.cc:1467] Processing CreateTablet for tablet 83deccf4879a4402a275375b85ff4c1c (DEFAULT_TABLE table=test-table [id=73969e760aa744608a97b1e9c9befc49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:49.953126 13516 tablet_service.cc:1467] Processing CreateTablet for tablet 83deccf4879a4402a275375b85ff4c1c (DEFAULT_TABLE table=test-table [id=73969e760aa744608a97b1e9c9befc49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:49.953315 13653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83deccf4879a4402a275375b85ff4c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:49.953398 13516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83deccf4879a4402a275375b85ff4c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:49.955453 13948 tablet_bootstrap.cc:492] T 83deccf4879a4402a275375b85ff4c1c P 10f2ba5d863c469c9084ea7824990d8d: Bootstrap starting.
I20260926 11:40:49.955763 13944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.947702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57138 (local address 127.6.96.3:37879)
0926 11:40:49.948025 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:49.948029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:49.948041 (+ 12us) server_negotiation.cc:407] Connection header received
0926 11:40:49.948086 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:49.948088 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:49.948132 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:49.948246 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:49.948486 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:49.949151 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:49.951681 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:49.952309 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:49.955160 (+ 2851us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:49.955225 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:49.955585 (+ 360us) server_negotiation.cc:299] Negotiation successful
0926 11:40:49.955663 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":66,"threads_started":1}
I20260926 11:40:49.955950 13947 tablet_bootstrap.cc:492] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c: Bootstrap starting.
I20260926 11:40:49.956197 13790 tablet_service.cc:1467] Processing CreateTablet for tablet 83deccf4879a4402a275375b85ff4c1c (DEFAULT_TABLE table=test-table [id=73969e760aa744608a97b1e9c9befc49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:49.956393 13790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83deccf4879a4402a275375b85ff4c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:49.956423 13948 tablet_bootstrap.cc:654] T 83deccf4879a4402a275375b85ff4c1c P 10f2ba5d863c469c9084ea7824990d8d: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:49.956758 13947 tablet_bootstrap.cc:654] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:49.957537 13948 tablet_bootstrap.cc:492] T 83deccf4879a4402a275375b85ff4c1c P 10f2ba5d863c469c9084ea7824990d8d: No bootstrap required, opened a new log
I20260926 11:40:49.957613 13948 ts_tablet_manager.cc:1397] T 83deccf4879a4402a275375b85ff4c1c P 10f2ba5d863c469c9084ea7824990d8d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 11:40:49.957912 13947 tablet_bootstrap.cc:492] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c: No bootstrap required, opened a new log
I20260926 11:40:49.957984 13947 ts_tablet_manager.cc:1397] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 11:40:49.958069 13948 raft_consensus.cc:348] T 83deccf4879a4402a275375b85ff4c1c P 10f2ba5d863c469c9084ea7824990d8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f2ba5d863c469c9084ea7824990d8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38253 } } peers { permanent_uuid: "c8733268bb584adbb5deeaac640b5e3f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37879 } } peers { permanent_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46263 } }
I20260926 11:40:49.958171 13948 raft_consensus.cc:374] T 83deccf4879a4402a275375b85ff4c1c P 10f2ba5d863c469c9084ea7824990d8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:49.958200 13948 raft_consensus.cc:729] T 83deccf4879a4402a275375b85ff4c1c P 10f2ba5d863c469c9084ea7824990d8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10f2ba5d863c469c9084ea7824990d8d, State: Initialized, Role: FOLLOWER
I20260926 11:40:49.958325 13948 consensus_queue.cc:260] T 83deccf4879a4402a275375b85ff4c1c P 10f2ba5d863c469c9084ea7824990d8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f2ba5d863c469c9084ea7824990d8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38253 } } peers { permanent_uuid: "c8733268bb584adbb5deeaac640b5e3f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37879 } } peers { permanent_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46263 } }
I20260926 11:40:49.958335 13947 raft_consensus.cc:348] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f2ba5d863c469c9084ea7824990d8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38253 } } peers { permanent_uuid: "c8733268bb584adbb5deeaac640b5e3f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37879 } } peers { permanent_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46263 } }
I20260926 11:40:49.958429 13947 raft_consensus.cc:374] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:49.958496 13947 raft_consensus.cc:729] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec00c3f1615d44b6adeedbe52d77f35c, State: Initialized, Role: FOLLOWER
I20260926 11:40:49.958649 13951 tablet_bootstrap.cc:492] T 83deccf4879a4402a275375b85ff4c1c P c8733268bb584adbb5deeaac640b5e3f: Bootstrap starting.
I20260926 11:40:49.958663 13948 ts_tablet_manager.cc:1428] T 83deccf4879a4402a275375b85ff4c1c P 10f2ba5d863c469c9084ea7824990d8d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:49.958640 13947 consensus_queue.cc:260] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f2ba5d863c469c9084ea7824990d8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38253 } } peers { permanent_uuid: "c8733268bb584adbb5deeaac640b5e3f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37879 } } peers { permanent_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46263 } }
I20260926 11:40:49.960335 13951 tablet_bootstrap.cc:654] T 83deccf4879a4402a275375b85ff4c1c P c8733268bb584adbb5deeaac640b5e3f: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:49.961086 13947 ts_tablet_manager.cc:1428] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260926 11:40:49.961369 13951 tablet_bootstrap.cc:492] T 83deccf4879a4402a275375b85ff4c1c P c8733268bb584adbb5deeaac640b5e3f: No bootstrap required, opened a new log
I20260926 11:40:49.961455 13951 ts_tablet_manager.cc:1397] T 83deccf4879a4402a275375b85ff4c1c P c8733268bb584adbb5deeaac640b5e3f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 11:40:49.961737 13956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.961305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.6.96.2:59403)
0926 11:40:49.961594 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:49.961650 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":86,"threads_started":1}
I20260926 11:40:49.961854 13951 raft_consensus.cc:348] T 83deccf4879a4402a275375b85ff4c1c P c8733268bb584adbb5deeaac640b5e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f2ba5d863c469c9084ea7824990d8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38253 } } peers { permanent_uuid: "c8733268bb584adbb5deeaac640b5e3f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37879 } } peers { permanent_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46263 } }
I20260926 11:40:49.961966 13951 raft_consensus.cc:374] T 83deccf4879a4402a275375b85ff4c1c P c8733268bb584adbb5deeaac640b5e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:49.962003 13951 raft_consensus.cc:729] T 83deccf4879a4402a275375b85ff4c1c P c8733268bb584adbb5deeaac640b5e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8733268bb584adbb5deeaac640b5e3f, State: Initialized, Role: FOLLOWER
I20260926 11:40:49.962127 13951 consensus_queue.cc:260] T 83deccf4879a4402a275375b85ff4c1c P c8733268bb584adbb5deeaac640b5e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f2ba5d863c469c9084ea7824990d8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38253 } } peers { permanent_uuid: "c8733268bb584adbb5deeaac640b5e3f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37879 } } peers { permanent_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46263 } }
I20260926 11:40:49.962447 13951 ts_tablet_manager.cc:1428] T 83deccf4879a4402a275375b85ff4c1c P c8733268bb584adbb5deeaac640b5e3f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:49.962600 13719 heartbeater.cc:503] Master 127.6.96.60:35405 was elected leader, sending a full tablet report...
I20260926 11:40:49.963447 13956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.962520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.6.96.2:37385)
0926 11:40:49.963232 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:49.963284 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":594}
I20260926 11:40:49.964792 13856 heartbeater.cc:503] Master 127.6.96.60:35405 was elected leader, sending a full tablet report...
I20260926 11:40:49.964993 13958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.963093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.6.96.3:54673)
0926 11:40:49.964825 (+ 1732us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:49.964894 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1668,"thread_start_us":88,"threads_started":1}
I20260926 11:40:49.965619 13958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.965414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.6.96.3:46747)
0926 11:40:49.965516 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:49.965563 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":28}
I20260926 11:40:49.965951 13582 heartbeater.cc:503] Master 127.6.96.60:35405 was elected leader, sending a full tablet report...
I20260926 11:40:49.966460 13953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.959171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.6.96.1:35011)
0926 11:40:49.966301 (+ 7130us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:49.966359 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7072,"thread_start_us":88,"threads_started":1}
I20260926 11:40:49.967455 13953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:49.967041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.6.96.1:40917)
0926 11:40:49.967332 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:49.967382 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":97,"threads_started":1}
I20260926 11:40:50.121351 13955 raft_consensus.cc:482] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:40:50.121501 13955 raft_consensus.cc:504] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f2ba5d863c469c9084ea7824990d8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38253 } } peers { permanent_uuid: "c8733268bb584adbb5deeaac640b5e3f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37879 } } peers { permanent_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46263 } }
I20260926 11:40:50.121876 13955 leader_election.cc:288] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10f2ba5d863c469c9084ea7824990d8d (127.6.96.1:38253), c8733268bb584adbb5deeaac640b5e3f (127.6.96.3:37879)
I20260926 11:40:50.122424 13810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83deccf4879a4402a275375b85ff4c1c" candidate_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8733268bb584adbb5deeaac640b5e3f" is_pre_election: true
I20260926 11:40:50.122412 13536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83deccf4879a4402a275375b85ff4c1c" candidate_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10f2ba5d863c469c9084ea7824990d8d" is_pre_election: true
I20260926 11:40:50.122567 13536 raft_consensus.cc:2454] T 83deccf4879a4402a275375b85ff4c1c P 10f2ba5d863c469c9084ea7824990d8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec00c3f1615d44b6adeedbe52d77f35c in term 0.
I20260926 11:40:50.122567 13810 raft_consensus.cc:2454] T 83deccf4879a4402a275375b85ff4c1c P c8733268bb584adbb5deeaac640b5e3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec00c3f1615d44b6adeedbe52d77f35c in term 0.
I20260926 11:40:50.122916 13609 leader_election.cc:302] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 10f2ba5d863c469c9084ea7824990d8d, c8733268bb584adbb5deeaac640b5e3f, ec00c3f1615d44b6adeedbe52d77f35c; no voters:
I20260926 11:40:50.123111 13955 raft_consensus.cc:2781] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:40:50.123169 13955 raft_consensus.cc:482] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:40:50.123198 13955 raft_consensus.cc:3037] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:50.124195 13955 raft_consensus.cc:504] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f2ba5d863c469c9084ea7824990d8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38253 } } peers { permanent_uuid: "c8733268bb584adbb5deeaac640b5e3f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37879 } } peers { permanent_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46263 } }
I20260926 11:40:50.124526 13955 leader_election.cc:288] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c [CANDIDATE]: Term 1 election: Requested vote from peers 10f2ba5d863c469c9084ea7824990d8d (127.6.96.1:38253), c8733268bb584adbb5deeaac640b5e3f (127.6.96.3:37879)
I20260926 11:40:50.124897 13536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83deccf4879a4402a275375b85ff4c1c" candidate_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10f2ba5d863c469c9084ea7824990d8d"
I20260926 11:40:50.124897 13810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83deccf4879a4402a275375b85ff4c1c" candidate_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8733268bb584adbb5deeaac640b5e3f"
I20260926 11:40:50.125046 13536 raft_consensus.cc:3037] T 83deccf4879a4402a275375b85ff4c1c P 10f2ba5d863c469c9084ea7824990d8d [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:50.125046 13810 raft_consensus.cc:3037] T 83deccf4879a4402a275375b85ff4c1c P c8733268bb584adbb5deeaac640b5e3f [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:50.126163 13810 raft_consensus.cc:2454] T 83deccf4879a4402a275375b85ff4c1c P c8733268bb584adbb5deeaac640b5e3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec00c3f1615d44b6adeedbe52d77f35c in term 1.
I20260926 11:40:50.126163 13536 raft_consensus.cc:2454] T 83deccf4879a4402a275375b85ff4c1c P 10f2ba5d863c469c9084ea7824990d8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec00c3f1615d44b6adeedbe52d77f35c in term 1.
I20260926 11:40:50.126475 13606 leader_election.cc:302] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8733268bb584adbb5deeaac640b5e3f, ec00c3f1615d44b6adeedbe52d77f35c; no voters:
I20260926 11:40:50.126638 13955 raft_consensus.cc:2781] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:50.126864 13955 raft_consensus.cc:686] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c [term 1 LEADER]: Becoming Leader. State: Replica: ec00c3f1615d44b6adeedbe52d77f35c, State: Running, Role: LEADER
I20260926 11:40:50.127013 13955 consensus_queue.cc:237] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f2ba5d863c469c9084ea7824990d8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38253 } } peers { permanent_uuid: "c8733268bb584adbb5deeaac640b5e3f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37879 } } peers { permanent_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46263 } }
I20260926 11:40:50.127537 13956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:50.127318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.6.96.2:35379)
0926 11:40:50.127414 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:50.127470 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260926 11:40:50.128739 13403 catalog_manager.cc:5495] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c reported cstate change: term changed from 0 to 1, leader changed from <none> to ec00c3f1615d44b6adeedbe52d77f35c (127.6.96.2). New cstate: current_term: 1 leader_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f2ba5d863c469c9084ea7824990d8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8733268bb584adbb5deeaac640b5e3f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec00c3f1615d44b6adeedbe52d77f35c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 46263 } health_report { overall_health: HEALTHY } } }
I20260926 11:40:50.142518 13946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:50.139562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49700 (local address 127.6.96.2:46263)
0926 11:40:50.139677 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:50.139681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:50.139708 (+ 27us) server_negotiation.cc:407] Connection header received
0926 11:40:50.139754 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:50.139756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:50.139803 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:50.139858 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:40:50.140160 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:50.140681 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:50.141515 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:50.141747 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:50.141839 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:50.142105 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:40:50.142180 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:40:50.142235 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:50.142357 (+ 122us) server_negotiation.cc:299] Negotiation successful
0926 11:40:50.142408 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 11:40:50.144714 13536 raft_consensus.cc:1261] T 83deccf4879a4402a275375b85ff4c1c P 10f2ba5d863c469c9084ea7824990d8d [term 1 FOLLOWER]: Refusing update from remote peer ec00c3f1615d44b6adeedbe52d77f35c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:50.144714 13810 raft_consensus.cc:1261] T 83deccf4879a4402a275375b85ff4c1c P c8733268bb584adbb5deeaac640b5e3f [term 1 FOLLOWER]: Refusing update from remote peer ec00c3f1615d44b6adeedbe52d77f35c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:50.145136 13960 consensus_queue.cc:1035] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8733268bb584adbb5deeaac640b5e3f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:50.145300 13955 consensus_queue.cc:1035] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c [LEADER]: Connected to new peer: Peer: permanent_uuid: "10f2ba5d863c469c9084ea7824990d8d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:50.146714 13959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:50.146412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.6.96.1:51765)
0926 11:40:50.146559 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:50.146619 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260926 11:40:50.148145 13958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:50.147898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.6.96.3:33887)
0926 11:40:50.148009 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:50.148069 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260926 11:40:50.150468 13965 mvcc.cc:205] Tried to move back new op lower bound from 7333571994188328960 to 7333571994121465856. Current Snapshot: MvccSnapshot[applied={T|T < 7333571994188328960}]
I20260926 11:40:50.150640 13956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:50.150253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.6.96.2:39263)
0926 11:40:50.150501 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:50.150563 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182}
I20260926 11:40:50.150942 13958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:50.150322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.6.96.3:41113)
0926 11:40:50.150799 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:50.150869 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412}
I20260926 11:40:50.151780 13956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:50.151553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.6.96.2:39553)
0926 11:40:50.151676 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:50.151720 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260926 11:40:50.154367 13402 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:50.154513 13402 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:50.155174 13959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:50.154944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44995 (local address 127.6.96.1:54599)
0926 11:40:50.155051 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:50.155104 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260926 11:40:50.158154 13402 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 51ff483a96c5450cb2b5cb4951b240a6: Sending DeleteTablet for 3 replicas of tablet 83deccf4879a4402a275375b85ff4c1c
I20260926 11:40:50.158740 13516 tablet_service.cc:1514] Processing DeleteTablet for tablet 83deccf4879a4402a275375b85ff4c1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:50 UTC) from {username='oryx', principal='oryx/127.6.96.60@KRBTEST.COM'} at 127.0.0.1:60348
I20260926 11:40:50.158842 13653 tablet_service.cc:1514] Processing DeleteTablet for tablet 83deccf4879a4402a275375b85ff4c1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:50 UTC) from {username='oryx', principal='oryx/127.6.96.60@KRBTEST.COM'} at 127.0.0.1:49690
I20260926 11:40:50.158960 13790 tablet_service.cc:1514] Processing DeleteTablet for tablet 83deccf4879a4402a275375b85ff4c1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:50 UTC) from {username='oryx', principal='oryx/127.6.96.60@KRBTEST.COM'} at 127.0.0.1:57138
I20260926 11:40:50.159101 13971 tablet_replica.cc:330] T 83deccf4879a4402a275375b85ff4c1c P 10f2ba5d863c469c9084ea7824990d8d: stopping tablet replica
I20260926 11:40:50.159229 13971 raft_consensus.cc:2229] T 83deccf4879a4402a275375b85ff4c1c P 10f2ba5d863c469c9084ea7824990d8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:50.159359 13972 tablet_replica.cc:330] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c: stopping tablet replica
I20260926 11:40:50.159365 13971 raft_consensus.cc:2258] T 83deccf4879a4402a275375b85ff4c1c P 10f2ba5d863c469c9084ea7824990d8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:50.159463 13972 raft_consensus.cc:2229] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c [term 1 LEADER]: Raft consensus shutting down.
I20260926 11:40:50.159626 13972 raft_consensus.cc:2258] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:50.159940 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 13365
I20260926 11:40:50.160641 13972 ts_tablet_manager.cc:1905] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:50.161134 13973 tablet_replica.cc:330] T 83deccf4879a4402a275375b85ff4c1c P c8733268bb584adbb5deeaac640b5e3f: stopping tablet replica
I20260926 11:40:50.161266 13973 raft_consensus.cc:2229] T 83deccf4879a4402a275375b85ff4c1c P c8733268bb584adbb5deeaac640b5e3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:50.161405 13973 raft_consensus.cc:2258] T 83deccf4879a4402a275375b85ff4c1c P c8733268bb584adbb5deeaac640b5e3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:50.162870 13973 ts_tablet_manager.cc:1905] T 83deccf4879a4402a275375b85ff4c1c P c8733268bb584adbb5deeaac640b5e3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:50.162957 13972 ts_tablet_manager.cc:1918] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:40:50.163028 13972 log.cc:1196] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-1/wal/wals/83deccf4879a4402a275375b85ff4c1c
I20260926 11:40:50.163292 13972 ts_tablet_manager.cc:1939] T 83deccf4879a4402a275375b85ff4c1c P ec00c3f1615d44b6adeedbe52d77f35c: Deleting consensus metadata
I20260926 11:40:50.163949 13390 catalog_manager.cc:4841] TS ec00c3f1615d44b6adeedbe52d77f35c (127.6.96.2:46263): tablet 83deccf4879a4402a275375b85ff4c1c (table test-table [id=73969e760aa744608a97b1e9c9befc49]) successfully deleted
I20260926 11:40:50.165117 13973 ts_tablet_manager.cc:1918] T 83deccf4879a4402a275375b85ff4c1c P c8733268bb584adbb5deeaac640b5e3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:40:50.165184 13973 log.cc:1196] T 83deccf4879a4402a275375b85ff4c1c P c8733268bb584adbb5deeaac640b5e3f: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/83deccf4879a4402a275375b85ff4c1c
I20260926 11:40:50.165410 13973 ts_tablet_manager.cc:1939] T 83deccf4879a4402a275375b85ff4c1c P c8733268bb584adbb5deeaac640b5e3f: Deleting consensus metadata
I20260926 11:40:50.166059 13387 catalog_manager.cc:4841] TS c8733268bb584adbb5deeaac640b5e3f (127.6.96.3:37879): tablet 83deccf4879a4402a275375b85ff4c1c (table test-table [id=73969e760aa744608a97b1e9c9befc49]) successfully deleted
I20260926 11:40:50.166808 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 13590
I20260926 11:40:50.167559 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:50.166912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:38253 (local address 127.0.0.1:60356)
0926 11:40:50.167416 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:50.167478 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.1:38253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425}
W20260926 11:40:50.168211 13390 catalog_manager.cc:4584] TS 10f2ba5d863c469c9084ea7824990d8d (127.6.96.1:38253): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 83deccf4879a4402a275375b85ff4c1c: Network error: Client connection negotiation failed: client connection to 127.6.96.1:38253: connect: Connection refused (error 111)
I20260926 11:40:50.173187 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 13726
I20260926 11:40:50.177996 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 13083
I20260926 11:40:50.182749 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 13185
2026-09-26T11:40:50Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5325 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:40:50 dist-test-slave-xxxx krb5kdc[13976](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:40:50 dist-test-slave-xxxx krb5kdc[13976](info): set up 2 sockets
Sep 26 11:40:50 dist-test-slave-xxxx krb5kdc[13976](info): commencing operation
krb5kdc: starting...
W20260926 11:40:51.131009 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:40:51 dist-test-slave-xxxx krb5kdc[13976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:40:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:40:51Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.62.keytab.
Entry for principal oryx/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.62.keytab.
I20260926 11:40:51.249337 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:39759
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.62.keytab
--principal=oryx/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:39685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:39759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:51.355635 13989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:51.355897 13989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:51.355952 13989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:51.360265 13989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:40:51.360325 13989 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 11:40:51.360347 13989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:51.360369 13989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:51.360399 13989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:40:51.360427 13989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:40:51.366147 13989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:39685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:39759
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.62.keytab
--principal=oryx/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:39759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:51.367300 13989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:51.368331 13989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:51.376621 13997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:51.376621 13994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:51.376780 13989 server_base.cc:1034] running on GCE node
W20260926 11:40:51.376646 13995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:51.377429 13989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:51.378494 13989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:51.379647 13989 hybrid_clock.cc:648] HybridClock initialized: now 1790422851379630 us; error 30 us; skew 500 ppm
Sep 26 11:40:51 dist-test-slave-xxxx krb5kdc[13976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422851, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:51.382272 13989 init.cc:377] Logged in from keytab as oryx/127.6.96.62@KRBTEST.COM (short username oryx)
I20260926 11:40:51.383162 13989 webserver.cc:460] Webserver started at http://127.6.96.62:45079/ using document root <none> and password file <none>
I20260926 11:40:51.383694 13989 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:51.383756 13989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:51.383935 13989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:51.385999 13989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "9cb6f434259e4f0d90652f2e1895b375"
format_stamp: "Formatted at 2026-09-26 11:40:51 on dist-test-slave-xxxx"
server_key: "10171e86d2f1588db82dd414f18d7f5f"
server_key_iv: "46479a8339dbebd05df821f07d45b2f5"
server_key_version: "encryptionkey@0"
I20260926 11:40:51.386551 13989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "9cb6f434259e4f0d90652f2e1895b375"
format_stamp: "Formatted at 2026-09-26 11:40:51 on dist-test-slave-xxxx"
server_key: "10171e86d2f1588db82dd414f18d7f5f"
server_key_iv: "46479a8339dbebd05df821f07d45b2f5"
server_key_version: "encryptionkey@0"
I20260926 11:40:51.390272 13989 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 11:40:51.392797 14004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:51.394011 13989 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:51.394124 13989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "9cb6f434259e4f0d90652f2e1895b375"
format_stamp: "Formatted at 2026-09-26 11:40:51 on dist-test-slave-xxxx"
server_key: "10171e86d2f1588db82dd414f18d7f5f"
server_key_iv: "46479a8339dbebd05df821f07d45b2f5"
server_key_version: "encryptionkey@0"
I20260926 11:40:51.394222 13989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:51.417762 13989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:51.420511 13989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:51.420684 13989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:51.427989 13989 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:39759
I20260926 11:40:51.428000 14066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:39759 every 8 connection(s)
I20260926 11:40:51.428896 13989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:40:51.431497 14067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:51.435673 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 13989
I20260926 11:40:51.435779 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:40:51.436069 6528 external_mini_cluster.cc:1428] Setting key 3a3d34acf8db72a79207fe3edba75575
I20260926 11:40:51.436911 14067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375: Bootstrap starting.
I20260926 11:40:51.439567 14067 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:51.440364 14067 log.cc:824] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:51.442296 14067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375: No bootstrap required, opened a new log
Sep 26 11:40:51 dist-test-slave-xxxx krb5kdc[13976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM
I20260926 11:40:51.446367 14067 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb6f434259e4f0d90652f2e1895b375" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 39759 } }
I20260926 11:40:51.446663 14067 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:51.446723 14067 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb6f434259e4f0d90652f2e1895b375, State: Initialized, Role: FOLLOWER
I20260926 11:40:51.447238 14067 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb6f434259e4f0d90652f2e1895b375" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 39759 } }
I20260926 11:40:51.447397 14067 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:40:51.447463 14067 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:40:51.447566 14067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:51.448602 14067 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb6f434259e4f0d90652f2e1895b375" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 39759 } }
I20260926 11:40:51.448951 14067 leader_election.cc:302] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9cb6f434259e4f0d90652f2e1895b375; no voters:
I20260926 11:40:51.449273 14067 leader_election.cc:288] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:40:51.449477 14072 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:51.449704 14072 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375 [term 1 LEADER]: Becoming Leader. State: Replica: 9cb6f434259e4f0d90652f2e1895b375, State: Running, Role: LEADER
I20260926 11:40:51.450026 14072 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb6f434259e4f0d90652f2e1895b375" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 39759 } }
I20260926 11:40:51.450496 14070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:51.437340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54644 (local address 127.6.96.62:39759)
0926 11:40:51.437759 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:51.437777 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:51.437816 (+ 39us) server_negotiation.cc:407] Connection header received
0926 11:40:51.438632 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:51.438656 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:51.439108 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:51.439453 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:51.440166 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:51.441473 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:51.442307 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:51.443000 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:51.443161 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:51.445666 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:51.445673 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:51.445690 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:51.445721 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:51.447565 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:51.447974 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:51.447980 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:51.447986 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:51.448126 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:51.448340 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:51.448343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:51.448345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:51.448848 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:51.449024 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:51.449311 (+ 287us) server_negotiation.cc:299] Negotiation successful
0926 11:40:51.449649 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":110,"threads_started":1}
I20260926 11:40:51.450605 14067 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:40:51.451665 14074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9cb6f434259e4f0d90652f2e1895b375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb6f434259e4f0d90652f2e1895b375" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 39759 } } }
I20260926 11:40:51.451782 14074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:51.451968 14074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9cb6f434259e4f0d90652f2e1895b375. Latest consensus state: current_term: 1 leader_uuid: "9cb6f434259e4f0d90652f2e1895b375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb6f434259e4f0d90652f2e1895b375" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 39759 } } }
I20260926 11:40:51.452033 14074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:51.452885 14081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:40:51.455578 14081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:40:51.460763 14081 catalog_manager.cc:1347] Generated new cluster ID: 9aefc5cc47fc48a1904df0ce3e1977d3
I20260926 11:40:51.460834 14081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:40:51.471998 14081 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:40:51.473284 14081 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:40:51.479105 14081 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375: Generated new TSK 0
I20260926 11:40:51.479910 14081 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.96.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.96.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.96.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.1.keytab.
Entry for principal oryx/127.6.96.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.1.keytab.
I20260926 11:40:51.510298 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:39759
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.1.keytab
--principal=oryx/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:39685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:51.618656 14093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:51.618935 14093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:51.618995 14093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:51.622989 14093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:51.623054 14093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:51.623117 14093 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 11:40:51.623174 14093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:40:51.628495 14093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:39685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.1.keytab
--principal=oryx/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.96.62:39759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:51.629586 14093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:51.630638 14093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:51.640038 14098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:51.640156 14093 server_base.cc:1034] running on GCE node
W20260926 11:40:51.640101 14099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:51.640038 14101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:51.640724 14093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:51.641366 14093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:51.642545 14093 hybrid_clock.cc:648] HybridClock initialized: now 1790422851642522 us; error 34 us; skew 500 ppm
Sep 26 11:40:51 dist-test-slave-xxxx krb5kdc[13976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422851, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:51.645183 14093 init.cc:377] Logged in from keytab as oryx/127.6.96.1@KRBTEST.COM (short username oryx)
I20260926 11:40:51.646147 14093 webserver.cc:460] Webserver started at http://127.6.96.1:36867/ using document root <none> and password file <none>
I20260926 11:40:51.646713 14093 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:51.646780 14093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:51.647010 14093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:51.649039 14093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "ee1c21f2176e4c1f8f0e41e7702136ac"
format_stamp: "Formatted at 2026-09-26 11:40:51 on dist-test-slave-xxxx"
server_key: "7c6500afc3cc9a314bf075ede6c50d41"
server_key_iv: "70981eeb2b4e1ee3bcbd52eff2803903"
server_key_version: "encryptionkey@0"
I20260926 11:40:51.649575 14093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "ee1c21f2176e4c1f8f0e41e7702136ac"
format_stamp: "Formatted at 2026-09-26 11:40:51 on dist-test-slave-xxxx"
server_key: "7c6500afc3cc9a314bf075ede6c50d41"
server_key_iv: "70981eeb2b4e1ee3bcbd52eff2803903"
server_key_version: "encryptionkey@0"
I20260926 11:40:51.653479 14093 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260926 11:40:51.655938 14108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:51.657121 14093 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 11:40:51.657228 14093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "ee1c21f2176e4c1f8f0e41e7702136ac"
format_stamp: "Formatted at 2026-09-26 11:40:51 on dist-test-slave-xxxx"
server_key: "7c6500afc3cc9a314bf075ede6c50d41"
server_key_iv: "70981eeb2b4e1ee3bcbd52eff2803903"
server_key_version: "encryptionkey@0"
I20260926 11:40:51.657320 14093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:51.671270 14093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:51.674418 14093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:51.674597 14093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:51.676070 14093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:51.676122 14093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:51.676177 14093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:51.676201 14093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 11:40:51 dist-test-slave-xxxx krb5kdc[13976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422851, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM
I20260926 11:40:51.685840 14093 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:42589
I20260926 11:40:51.685984 14223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:42589 every 8 connection(s)
I20260926 11:40:51.686897 14093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:40:51.688980 14070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:51.676938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:33339 (local address 127.6.96.62:39759)
0926 11:40:51.677081 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:51.677086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:51.677895 (+ 809us) server_negotiation.cc:407] Connection header received
0926 11:40:51.679041 (+ 1146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:51.679045 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:51.679106 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:51.679205 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:51.680438 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:51.680953 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:51.682647 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:51.683062 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:51.683163 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:51.685513 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:51.685517 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:51.685520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:51.685548 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:51.687024 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:51.687570 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:51.687574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:51.687576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:51.687661 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:51.687960 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:51.687963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:51.687965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:51.688152 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:51.688242 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:51.688698 (+ 456us) server_negotiation.cc:299] Negotiation successful
0926 11:40:51.688819 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 11:40:51.689707 14117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:51.677308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:39759 (local address 127.6.96.1:33339)
0926 11:40:51.677737 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:51.677784 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:51.678789 (+ 1005us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:51.679387 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:51.679397 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:51.679751 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:51.680284 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:51.680297 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:51.681069 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:51.681073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:51.682513 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:51.682526 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:51.683217 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:51.683221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:51.683376 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:51.683973 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:51.683993 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:51.685354 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:51.687130 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:51.687135 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:51.687149 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:51.687456 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:51.687741 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:51.687745 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:51.687747 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:51.687872 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:51.688251 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:51.688258 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:51.688479 (+ 221us) client_negotiation.cc:769] Sending connection context
0926 11:40:51.688640 (+ 161us) client_negotiation.cc:240] Negotiation successful
0926 11:40:51.688941 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":107,"threads_started":1}
I20260926 11:40:51.690881 14224 heartbeater.cc:347] Connected to a master server at 127.6.96.62:39759
I20260926 11:40:51.691011 14224 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:51.691464 14224 heartbeater.cc:511] Master 127.6.96.62:39759 requested a full tablet report, sending...
I20260926 11:40:51.692920 14020 ts_manager.cc:194] Registered new tserver with Master: ee1c21f2176e4c1f8f0e41e7702136ac (127.6.96.1:42589)
I20260926 11:40:51.694037 14020 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.96.1@KRBTEST.COM'} at 127.6.96.1:33339
I20260926 11:40:51.696475 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 14093
I20260926 11:40:51.696590 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:40:51.696908 6528 external_mini_cluster.cc:1428] Setting key 564f2a85e9e6b01b61da5fc7ccef276b
I20260926 11:40:51.699640 14070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:51.692933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:36893 (local address 127.6.96.62:39759)
0926 11:40:51.693051 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:51.693056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:51.693223 (+ 167us) server_negotiation.cc:407] Connection header received
0926 11:40:51.693317 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:51.693320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:51.693361 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:51.693424 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:51.693847 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:51.694395 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:51.695304 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:51.695510 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:51.695591 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:51.696629 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:51.696633 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:51.696636 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:51.696659 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:51.698186 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:51.698574 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:51.698577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:51.698579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:51.698647 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:51.698916 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:51.698919 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:51.698921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:51.699084 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:51.699169 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:51.699395 (+ 226us) server_negotiation.cc:299] Negotiation successful
0926 11:40:51.699489 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 11:40:51.699648 14230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:51.692879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:39759 (local address 127.6.96.1:36893)
0926 11:40:51.693150 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:51.693161 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:51.693248 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:51.693450 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:51.693452 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:51.693658 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:51.693747 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:51.693752 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:51.694513 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:51.694517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:51.695202 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:51.695210 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:51.695601 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:51.695605 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:51.695688 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:51.696161 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:51.696171 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:51.696462 (+ 291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:51.698281 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:51.698284 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:51.698285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:51.698460 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:51.698724 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:51.698745 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:51.698747 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:51.698846 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:51.699171 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:51.699173 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:51.699256 (+ 83us) client_negotiation.cc:769] Sending connection context
0926 11:40:51.699338 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 11:40:51.699469 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":94,"threads_started":1}
WARNING: no policy specified for oryx/127.6.96.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.96.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.96.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.2.keytab.
Entry for principal oryx/127.6.96.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.2.keytab.
I20260926 11:40:51.721576 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:39759
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.2.keytab
--principal=oryx/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:39685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:51.827340 14233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:51.827610 14233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:51.827669 14233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:51.831652 14233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:51.831724 14233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:51.831777 14233 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 11:40:51.831830 14233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:40:51.837203 14233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:39685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.2.keytab
--principal=oryx/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.96.62:39759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:51.838292 14233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:51.839340 14233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:51.848382 14238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:51.848369 14241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:51.848394 14239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:51.848594 14233 server_base.cc:1034] running on GCE node
I20260926 11:40:51.848982 14233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:51.849660 14233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:51.850807 14233 hybrid_clock.cc:648] HybridClock initialized: now 1790422851850785 us; error 34 us; skew 500 ppm
Sep 26 11:40:51 dist-test-slave-xxxx krb5kdc[13976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422851, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:51.853477 14233 init.cc:377] Logged in from keytab as oryx/127.6.96.2@KRBTEST.COM (short username oryx)
I20260926 11:40:51.854322 14233 webserver.cc:460] Webserver started at http://127.6.96.2:35143/ using document root <none> and password file <none>
I20260926 11:40:51.854887 14233 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:51.854961 14233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:51.855154 14233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:51.857165 14233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "1d08dd932879485aabf14c8b3909a7b9"
format_stamp: "Formatted at 2026-09-26 11:40:51 on dist-test-slave-xxxx"
server_key: "246944069ea03ff1b97fd671044470b3"
server_key_iv: "77a78813350b3e046a193191470adf10"
server_key_version: "encryptionkey@0"
I20260926 11:40:51.857731 14233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "1d08dd932879485aabf14c8b3909a7b9"
format_stamp: "Formatted at 2026-09-26 11:40:51 on dist-test-slave-xxxx"
server_key: "246944069ea03ff1b97fd671044470b3"
server_key_iv: "77a78813350b3e046a193191470adf10"
server_key_version: "encryptionkey@0"
I20260926 11:40:51.861451 14233 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260926 11:40:51.863931 14248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:51.865183 14233 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260926 11:40:51.865291 14233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "1d08dd932879485aabf14c8b3909a7b9"
format_stamp: "Formatted at 2026-09-26 11:40:51 on dist-test-slave-xxxx"
server_key: "246944069ea03ff1b97fd671044470b3"
server_key_iv: "77a78813350b3e046a193191470adf10"
server_key_version: "encryptionkey@0"
I20260926 11:40:51.865388 14233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:51.889381 14233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:51.892580 14233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:51.892750 14233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:51.894204 14233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:51.894254 14233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:51.894307 14233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:51.894335 14233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 11:40:51 dist-test-slave-xxxx krb5kdc[13976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422851, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM
I20260926 11:40:51.904136 14233 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:42263
I20260926 11:40:51.904394 14363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:42263 every 8 connection(s)
I20260926 11:40:51.905205 14233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:40:51.906859 14070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:51.894949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:38813 (local address 127.6.96.62:39759)
0926 11:40:51.895106 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:51.895111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:51.895971 (+ 860us) server_negotiation.cc:407] Connection header received
0926 11:40:51.896986 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:51.896990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:51.897044 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:51.897119 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:51.898617 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:51.899382 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:51.900603 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:51.900827 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:51.900915 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:51.903270 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:51.903274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:51.903277 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:51.903301 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:51.904904 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:51.905370 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:51.905374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:51.905376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:51.905455 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:51.905838 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:51.905842 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:51.905844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:51.906069 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:51.906147 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:51.906606 (+ 459us) server_negotiation.cc:299] Negotiation successful
0926 11:40:51.906713 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 11:40:51.907398 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 14233
I20260926 11:40:51.907531 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:40:51.907711 14257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:51.895381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:39759 (local address 127.6.96.2:38813)
0926 11:40:51.895814 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:51.895862 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:51.896750 (+ 888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:51.897541 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:51.897550 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:51.897931 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:51.898489 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:51.898507 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:51.899491 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:51.899495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:51.900483 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:51.900493 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:51.900974 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:51.900979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:51.901146 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:51.901678 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:51.901694 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:51.903119 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:51.905020 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:51.905025 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:51.905036 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:51.905290 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:51.905544 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:51.905549 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:51.905551 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:51.905720 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:51.906160 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:51.906165 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:51.906379 (+ 214us) client_negotiation.cc:769] Sending connection context
0926 11:40:51.906527 (+ 148us) client_negotiation.cc:240] Negotiation successful
0926 11:40:51.906764 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":105,"threads_started":1}
I20260926 11:40:51.907838 6528 external_mini_cluster.cc:1428] Setting key 0e436e2cb48a15db9355fc5b2e6e5a99
I20260926 11:40:51.909119 14364 heartbeater.cc:347] Connected to a master server at 127.6.96.62:39759
I20260926 11:40:51.909248 14364 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:51.909767 14364 heartbeater.cc:511] Master 127.6.96.62:39759 requested a full tablet report, sending...
I20260926 11:40:51.911406 14020 ts_manager.cc:194] Registered new tserver with Master: 1d08dd932879485aabf14c8b3909a7b9 (127.6.96.2:42263)
I20260926 11:40:51.912134 14020 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.96.2@KRBTEST.COM'} at 127.6.96.2:38813
WARNING: no policy specified for oryx/127.6.96.3@KRBTEST.COM; defaulting to no policy
I20260926 11:40:51.917474 14070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:51.911044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:58559 (local address 127.6.96.62:39759)
0926 11:40:51.911261 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:51.911265 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:51.911312 (+ 47us) server_negotiation.cc:407] Connection header received
0926 11:40:51.911461 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:51.911464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:51.911507 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:51.911589 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:51.912026 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:51.912570 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:51.913351 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:51.913577 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:51.913657 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:51.914704 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:51.914707 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:51.914710 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:51.914733 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:51.915977 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:51.916334 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:51.916338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:51.916340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:51.916447 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:51.916681 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:51.916684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:51.916686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:51.916895 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:51.916980 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:51.917212 (+ 232us) server_negotiation.cc:299] Negotiation successful
0926 11:40:51.917310 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":151}
I20260926 11:40:51.917480 14371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:51.910946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:39759 (local address 127.6.96.2:58559)
0926 11:40:51.911238 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:51.911251 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:51.911370 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:51.911601 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:51.911604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:51.911809 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:51.911921 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:51.911928 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:51.912687 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:51.912689 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:51.913248 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:51.913255 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:51.913670 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:51.913674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:51.913786 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:51.914266 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:51.914277 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:51.914585 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:51.916070 (+ 1485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:51.916073 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:51.916076 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:51.916255 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:51.916526 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:51.916529 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:51.916531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:51.916609 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:51.916981 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:51.916984 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:51.917071 (+ 87us) client_negotiation.cc:769] Sending connection context
0926 11:40:51.917153 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 11:40:51.917288 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":1,"thread_start_us":89,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.96.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.96.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.3.keytab.
Entry for principal oryx/127.6.96.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.3.keytab.
I20260926 11:40:51.935401 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:39759
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.3.keytab
--principal=oryx/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:39685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:52.042529 14373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:52.042791 14373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:52.042870 14373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:52.046811 14373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:52.046900 14373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:52.046941 14373 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 11:40:52.046976 14373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:40:52.052222 14373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:39685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.3.keytab
--principal=oryx/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.96.62:39759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:52.053254 14373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:52.054206 14373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 11:40:52.063632 14373 server_base.cc:1034] running on GCE node
W20260926 11:40:52.063593 14378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:52.063604 14379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:52.063593 14381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:52.064342 14373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:52.064954 14373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:52.066111 14373 hybrid_clock.cc:648] HybridClock initialized: now 1790422852066081 us; error 43 us; skew 500 ppm
Sep 26 11:40:52 dist-test-slave-xxxx krb5kdc[13976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422852, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:52.068799 14373 init.cc:377] Logged in from keytab as oryx/127.6.96.3@KRBTEST.COM (short username oryx)
I20260926 11:40:52.069615 14373 webserver.cc:460] Webserver started at http://127.6.96.3:39889/ using document root <none> and password file <none>
I20260926 11:40:52.070094 14373 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:52.070148 14373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:52.070310 14373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:52.072330 14373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "039d3bd43cdd4aa382e95b24e2401a1e"
format_stamp: "Formatted at 2026-09-26 11:40:52 on dist-test-slave-xxxx"
server_key: "866d969ff6b548bad2f21e070e2ef658"
server_key_iv: "89599ddc648f9c28d3a79abd6990d75f"
server_key_version: "encryptionkey@0"
I20260926 11:40:52.072808 14373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "039d3bd43cdd4aa382e95b24e2401a1e"
format_stamp: "Formatted at 2026-09-26 11:40:52 on dist-test-slave-xxxx"
server_key: "866d969ff6b548bad2f21e070e2ef658"
server_key_iv: "89599ddc648f9c28d3a79abd6990d75f"
server_key_version: "encryptionkey@0"
I20260926 11:40:52.076278 14373 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260926 11:40:52.078613 14388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:52.079828 14373 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:52.079921 14373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "039d3bd43cdd4aa382e95b24e2401a1e"
format_stamp: "Formatted at 2026-09-26 11:40:52 on dist-test-slave-xxxx"
server_key: "866d969ff6b548bad2f21e070e2ef658"
server_key_iv: "89599ddc648f9c28d3a79abd6990d75f"
server_key_version: "encryptionkey@0"
I20260926 11:40:52.080024 14373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:52.091858 14373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:52.094980 14373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:52.095148 14373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:52.096529 14373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:52.096580 14373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:52.096632 14373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:52.096659 14373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:52.106207 14373 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:37865
I20260926 11:40:52.106247 14503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:37865 every 8 connection(s)
Sep 26 11:40:52 dist-test-slave-xxxx krb5kdc[13976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422852, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM
I20260926 11:40:52.107183 14373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:40:52.110249 14070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.097327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:49667 (local address 127.6.96.62:39759)
0926 11:40:52.097431 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:52.097435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:52.098574 (+ 1139us) server_negotiation.cc:407] Connection header received
0926 11:40:52.099761 (+ 1187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:52.099764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:52.099821 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:52.099892 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:52.101341 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.102150 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.103798 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.104139 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.104248 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.106895 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:52.106899 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:52.106902 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:52.106929 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:52.108519 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:52.108954 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:52.108956 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:52.108958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:52.109025 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:52.109291 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:52.109292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:52.109294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:52.109461 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:52.109527 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:52.110029 (+ 502us) server_negotiation.cc:299] Negotiation successful
0926 11:40:52.110133 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 11:40:52.111063 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 14373
I20260926 11:40:52.111155 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:40:52.111136 14397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.097926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:39759 (local address 127.6.96.3:49667)
0926 11:40:52.098401 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:52.098479 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:52.099529 (+ 1050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:52.100098 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:52.100107 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:52.100508 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:52.101176 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:52.101196 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.102289 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.102294 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:52.103687 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:52.103697 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.104283 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.104287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:52.104473 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.105123 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:52.105142 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:52.106706 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:52.108615 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:52.108619 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:52.108630 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:52.108880 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:52.109087 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:52.109089 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:52.109090 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:52.109232 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:52.109564 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:52.109569 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:52.109796 (+ 227us) client_negotiation.cc:769] Sending connection context
0926 11:40:52.109994 (+ 198us) client_negotiation.cc:240] Negotiation successful
0926 11:40:52.110258 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":140,"threads_started":1}
I20260926 11:40:52.111410 6528 external_mini_cluster.cc:1428] Setting key ac47bcb5dc9f6290f8d8342d2404dc72
I20260926 11:40:52.114131 14504 heartbeater.cc:347] Connected to a master server at 127.6.96.62:39759
I20260926 11:40:52.114270 14504 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:52.114699 14504 heartbeater.cc:511] Master 127.6.96.62:39759 requested a full tablet report, sending...
I20260926 11:40:52.115620 14019 ts_manager.cc:194] Registered new tserver with Master: 039d3bd43cdd4aa382e95b24e2401a1e (127.6.96.3:37865)
I20260926 11:40:52.116191 14019 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.96.3@KRBTEST.COM'} at 127.6.96.3:49667
I20260926 11:40:52.120934 14070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.114345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:59981 (local address 127.6.96.62:39759)
0926 11:40:52.114440 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:52.114444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:52.115079 (+ 635us) server_negotiation.cc:407] Connection header received
0926 11:40:52.115189 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:52.115192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:52.115234 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:52.115301 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:52.115740 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.116250 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.117090 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.117414 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.117527 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.118497 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:52.118500 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:52.118503 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:52.118523 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:52.119789 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:52.120095 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:52.120097 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:52.120104 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:52.120167 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:52.120340 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:52.120342 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:52.120343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:52.120493 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:52.120559 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:52.120745 (+ 186us) server_negotiation.cc:299] Negotiation successful
0926 11:40:52.120829 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 11:40:52.120940 14510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.114295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:39759 (local address 127.6.96.3:59981)
0926 11:40:52.115003 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:52.115014 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:52.115110 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:52.115315 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:52.115317 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:52.115503 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:52.115624 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:52.115632 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.116381 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.116384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:52.116982 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:52.116990 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.117518 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.117520 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:52.117580 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.118072 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:52.118081 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:52.118382 (+ 301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:52.119863 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:52.119866 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:52.119868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:52.120029 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:52.120224 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:52.120226 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:52.120227 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:52.120285 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:52.120551 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:52.120553 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:52.120633 (+ 80us) client_negotiation.cc:769] Sending connection context
0926 11:40:52.120703 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 11:40:52.120801 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":92,"threads_started":1}
I20260926 11:40:52.124684 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 11:40:52 dist-test-slave-xxxx krb5kdc[13976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422852, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 11:40:52 dist-test-slave-xxxx krb5kdc[13976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:40:52 dist-test-slave-xxxx krb5kdc[13976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
W20260926 11:40:52.146456 14070 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:52.140588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54646 (local address 127.6.96.62:39759)
0926 11:40:52.140705 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:52.140708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:52.140719 (+ 11us) server_negotiation.cc:407] Connection header received
0926 11:40:52.140757 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:52.140760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:52.140799 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:52.140861 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:52.141580 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.142094 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.142956 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.143176 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.143246 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.146322 (+ 3076us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54646: BlockingRecv error: recv got EOF from 127.0.0.1:54646 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 26 11:40:52 dist-test-slave-xxxx krb5kdc[13976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422852, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM
I20260926 11:40:52.155182 14070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.147810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54650 (local address 127.6.96.62:39759)
0926 11:40:52.147913 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:52.147916 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:52.148034 (+ 118us) server_negotiation.cc:407] Connection header received
0926 11:40:52.148130 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:52.148132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:52.148169 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:52.148234 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:52.148743 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.149238 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.150004 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.150263 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.150345 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.152565 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:52.152568 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:52.152570 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:52.152593 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:52.153835 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:52.154193 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:52.154196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:52.154197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:52.154263 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:52.154443 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:52.154445 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:52.154446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:52.154619 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:52.154712 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:52.154953 (+ 241us) server_negotiation.cc:299] Negotiation successful
0926 11:40:52.155044 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 11:40:52.157908 14019 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 11:40:52.160804 14019 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:40:52.174369 14526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.168839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:42263 (local address 127.0.0.1:55486)
0926 11:40:52.169279 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:52.169329 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:52.169444 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:52.170616 (+ 1172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:52.170620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:52.170636 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:52.170718 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:52.170724 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.172085 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.172096 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:52.173208 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:52.173215 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.173968 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.173972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:52.174072 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.174106 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 11:40:52.174172 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 11:40:52.174254 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":39,"thread_start_us":86,"threads_started":1}
I20260926 11:40:52.174602 14529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.168907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:37865 (local address 127.0.0.1:38152)
0926 11:40:52.169776 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:52.169789 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:52.169875 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:52.170474 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:52.170479 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:52.170506 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:52.170625 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:52.170630 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.171716 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.171719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:52.173018 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:52.173028 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.174212 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.174216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:52.174304 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.174325 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 11:40:52.174395 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 11:40:52.174468 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"mutex_wait_us":27,"thread_start_us":74,"threads_started":1}
I20260926 11:40:52.174672 14530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.169627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38152 (local address 127.6.96.3:37865)
0926 11:40:52.170061 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:52.170073 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:52.170105 (+ 32us) server_negotiation.cc:407] Connection header received
0926 11:40:52.170173 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:52.170177 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:52.170330 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:52.170473 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:52.170718 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.171634 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.173119 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.173909 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.173993 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.174155 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:52.174448 (+ 293us) server_negotiation.cc:299] Negotiation successful
0926 11:40:52.174550 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":108,"threads_started":1}
I20260926 11:40:52.175356 14528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.168990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55486 (local address 127.6.96.2:42263)
0926 11:40:52.170187 (+ 1197us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:52.170197 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:52.170217 (+ 20us) server_negotiation.cc:407] Connection header received
0926 11:40:52.170279 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:52.170283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:52.170479 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:52.170609 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:52.170857 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.171975 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.173309 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.173843 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.174977 (+ 1134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.175101 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:52.175149 (+ 48us) server_negotiation.cc:299] Negotiation successful
0926 11:40:52.175242 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1112,"thread_start_us":119,"threads_started":1}
I20260926 11:40:52.175446 14527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.169426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:42589 (local address 127.0.0.1:56996)
0926 11:40:52.170818 (+ 1392us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:52.170856 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:52.170934 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:52.171408 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:52.171411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:52.171429 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:52.171524 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:52.171529 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.172719 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.172722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:52.173868 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:52.173880 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.175082 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.175086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:52.175177 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.175198 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 11:40:52.175266 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 11:40:52.175329 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1335,"mutex_wait_us":3,"thread_start_us":109,"threads_started":1}
I20260926 11:40:52.177712 14294 tablet_service.cc:1467] Processing CreateTablet for tablet 4322f45245db4b9e8579d7c98be94ac5 (DEFAULT_TABLE table=test-table [id=7293e4b9be3e4f7792c6559c92ce0d03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:52.177711 14438 tablet_service.cc:1467] Processing CreateTablet for tablet 4322f45245db4b9e8579d7c98be94ac5 (DEFAULT_TABLE table=test-table [id=7293e4b9be3e4f7792c6559c92ce0d03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:52.178617 14438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4322f45245db4b9e8579d7c98be94ac5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:52.178618 14294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4322f45245db4b9e8579d7c98be94ac5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:52.179033 14531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.169821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56996 (local address 127.6.96.1:42589)
0926 11:40:52.170148 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:52.170160 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:52.171030 (+ 870us) server_negotiation.cc:407] Connection header received
0926 11:40:52.171105 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:52.171110 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:52.171277 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:52.171411 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:52.171623 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.172610 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.174013 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.174745 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.174869 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.175027 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:52.175735 (+ 708us) server_negotiation.cc:299] Negotiation successful
0926 11:40:52.178889 (+ 3154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":73,"threads_started":1}
I20260926 11:40:52.181802 14158 tablet_service.cc:1467] Processing CreateTablet for tablet 4322f45245db4b9e8579d7c98be94ac5 (DEFAULT_TABLE table=test-table [id=7293e4b9be3e4f7792c6559c92ce0d03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:52.182884 14158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4322f45245db4b9e8579d7c98be94ac5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:52.185160 14532 tablet_bootstrap.cc:492] T 4322f45245db4b9e8579d7c98be94ac5 P 039d3bd43cdd4aa382e95b24e2401a1e: Bootstrap starting.
I20260926 11:40:52.187321 14532 tablet_bootstrap.cc:654] T 4322f45245db4b9e8579d7c98be94ac5 P 039d3bd43cdd4aa382e95b24e2401a1e: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:52.187534 14533 tablet_bootstrap.cc:492] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9: Bootstrap starting.
I20260926 11:40:52.188257 14532 log.cc:824] T 4322f45245db4b9e8579d7c98be94ac5 P 039d3bd43cdd4aa382e95b24e2401a1e: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:52.189728 14533 tablet_bootstrap.cc:654] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:52.190032 14535 tablet_bootstrap.cc:492] T 4322f45245db4b9e8579d7c98be94ac5 P ee1c21f2176e4c1f8f0e41e7702136ac: Bootstrap starting.
I20260926 11:40:52.190369 14532 tablet_bootstrap.cc:492] T 4322f45245db4b9e8579d7c98be94ac5 P 039d3bd43cdd4aa382e95b24e2401a1e: No bootstrap required, opened a new log
I20260926 11:40:52.190650 14532 ts_tablet_manager.cc:1397] T 4322f45245db4b9e8579d7c98be94ac5 P 039d3bd43cdd4aa382e95b24e2401a1e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 11:40:52.190667 14533 log.cc:824] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:52.192080 14535 tablet_bootstrap.cc:654] T 4322f45245db4b9e8579d7c98be94ac5 P ee1c21f2176e4c1f8f0e41e7702136ac: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:52.192776 14535 log.cc:824] T 4322f45245db4b9e8579d7c98be94ac5 P ee1c21f2176e4c1f8f0e41e7702136ac: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:52.192829 14533 tablet_bootstrap.cc:492] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9: No bootstrap required, opened a new log
I20260926 11:40:52.193019 14533 ts_tablet_manager.cc:1397] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 11:40:52.194586 14535 tablet_bootstrap.cc:492] T 4322f45245db4b9e8579d7c98be94ac5 P ee1c21f2176e4c1f8f0e41e7702136ac: No bootstrap required, opened a new log
I20260926 11:40:52.194756 14535 ts_tablet_manager.cc:1397] T 4322f45245db4b9e8579d7c98be94ac5 P ee1c21f2176e4c1f8f0e41e7702136ac: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 11:40:52.194591 14532 raft_consensus.cc:348] T 4322f45245db4b9e8579d7c98be94ac5 P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } } peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } }
I20260926 11:40:52.194905 14532 raft_consensus.cc:374] T 4322f45245db4b9e8579d7c98be94ac5 P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:52.194973 14532 raft_consensus.cc:729] T 4322f45245db4b9e8579d7c98be94ac5 P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039d3bd43cdd4aa382e95b24e2401a1e, State: Initialized, Role: FOLLOWER
I20260926 11:40:52.195427 14532 consensus_queue.cc:260] T 4322f45245db4b9e8579d7c98be94ac5 P 039d3bd43cdd4aa382e95b24e2401a1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } } peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } }
I20260926 11:40:52.196115 14504 heartbeater.cc:503] Master 127.6.96.62:39759 was elected leader, sending a full tablet report...
I20260926 11:40:52.196437 14532 ts_tablet_manager.cc:1428] T 4322f45245db4b9e8579d7c98be94ac5 P 039d3bd43cdd4aa382e95b24e2401a1e: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260926 11:40:52.196996 14533 raft_consensus.cc:348] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } } peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } }
I20260926 11:40:52.197288 14533 raft_consensus.cc:374] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:52.197347 14533 raft_consensus.cc:729] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d08dd932879485aabf14c8b3909a7b9, State: Initialized, Role: FOLLOWER
I20260926 11:40:52.197738 14533 consensus_queue.cc:260] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } } peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } }
I20260926 11:40:52.198365 14364 heartbeater.cc:503] Master 127.6.96.62:39759 was elected leader, sending a full tablet report...
I20260926 11:40:52.198205 14535 raft_consensus.cc:348] T 4322f45245db4b9e8579d7c98be94ac5 P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } } peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } }
I20260926 11:40:52.198446 14535 raft_consensus.cc:374] T 4322f45245db4b9e8579d7c98be94ac5 P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:52.198487 14535 raft_consensus.cc:729] T 4322f45245db4b9e8579d7c98be94ac5 P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee1c21f2176e4c1f8f0e41e7702136ac, State: Initialized, Role: FOLLOWER
I20260926 11:40:52.198769 14533 ts_tablet_manager.cc:1428] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 11:40:52.198954 14535 consensus_queue.cc:260] T 4322f45245db4b9e8579d7c98be94ac5 P ee1c21f2176e4c1f8f0e41e7702136ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } } peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } }
I20260926 11:40:52.199573 14224 heartbeater.cc:503] Master 127.6.96.62:39759 was elected leader, sending a full tablet report...
I20260926 11:40:52.200207 14535 ts_tablet_manager.cc:1428] T 4322f45245db4b9e8579d7c98be94ac5 P ee1c21f2176e4c1f8f0e41e7702136ac: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 11:40:52.275223 14539 raft_consensus.cc:482] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:40:52.275444 14539 raft_consensus.cc:504] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } } peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } }
I20260926 11:40:52.276588 14539 leader_election.cc:288] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039d3bd43cdd4aa382e95b24e2401a1e (127.6.96.3:37865), ee1c21f2176e4c1f8f0e41e7702136ac (127.6.96.1:42589)
I20260926 11:40:52.280480 14530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.276915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:35029 (local address 127.6.96.3:37865)
0926 11:40:52.277036 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:52.277040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:52.277335 (+ 295us) server_negotiation.cc:407] Connection header received
0926 11:40:52.277462 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:52.277464 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:52.277515 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:52.277586 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:52.277833 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.278342 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.279522 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.279943 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.280018 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.280068 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:52.280290 (+ 222us) server_negotiation.cc:299] Negotiation successful
0926 11:40:52.280362 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 11:40:52.280579 14531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.276920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:48929 (local address 127.6.96.1:42589)
0926 11:40:52.277036 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:52.277040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:52.277107 (+ 67us) server_negotiation.cc:407] Connection header received
0926 11:40:52.277234 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:52.277237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:52.277294 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:52.277365 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:52.277612 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.278193 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.279645 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.280069 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.280192 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.280248 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:52.280402 (+ 154us) server_negotiation.cc:299] Negotiation successful
0926 11:40:52.280470 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 11:40:52.280527 14541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.276816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:37865 (local address 127.6.96.2:35029)
0926 11:40:52.277231 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:52.277245 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:52.277357 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:52.277602 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:52.277604 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:52.277620 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:52.277734 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:52.277740 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.278513 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.278516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:52.279384 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:52.279394 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.280052 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.280056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:52.280159 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.280196 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 11:40:52.280265 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 11:40:52.280328 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":44,"thread_start_us":111,"threads_started":1}
I20260926 11:40:52.280565 14257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.276816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:42589 (local address 127.6.96.2:48929)
0926 11:40:52.277015 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:52.277030 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:52.277134 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:52.277381 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:52.277383 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:52.277397 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:52.277519 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:52.277525 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.278299 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.278303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:52.279528 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:52.279538 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.280199 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.280202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:52.280286 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.280308 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 11:40:52.280375 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 11:40:52.280440 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260926 11:40:52.281220 14458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4322f45245db4b9e8579d7c98be94ac5" candidate_uuid: "1d08dd932879485aabf14c8b3909a7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" is_pre_election: true
I20260926 11:40:52.281294 14178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4322f45245db4b9e8579d7c98be94ac5" candidate_uuid: "1d08dd932879485aabf14c8b3909a7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" is_pre_election: true
I20260926 11:40:52.281500 14458 raft_consensus.cc:2454] T 4322f45245db4b9e8579d7c98be94ac5 P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d08dd932879485aabf14c8b3909a7b9 in term 0.
I20260926 11:40:52.281546 14178 raft_consensus.cc:2454] T 4322f45245db4b9e8579d7c98be94ac5 P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d08dd932879485aabf14c8b3909a7b9 in term 0.
I20260926 11:40:52.281993 14250 leader_election.cc:302] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d08dd932879485aabf14c8b3909a7b9, ee1c21f2176e4c1f8f0e41e7702136ac; no voters:
I20260926 11:40:52.282235 14539 raft_consensus.cc:2781] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:40:52.282315 14539 raft_consensus.cc:482] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:40:52.282361 14539 raft_consensus.cc:3037] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:52.283443 14539 raft_consensus.cc:504] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } } peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } }
I20260926 11:40:52.283759 14539 leader_election.cc:288] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9 [CANDIDATE]: Term 1 election: Requested vote from peers 039d3bd43cdd4aa382e95b24e2401a1e (127.6.96.3:37865), ee1c21f2176e4c1f8f0e41e7702136ac (127.6.96.1:42589)
I20260926 11:40:52.284090 14458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4322f45245db4b9e8579d7c98be94ac5" candidate_uuid: "1d08dd932879485aabf14c8b3909a7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039d3bd43cdd4aa382e95b24e2401a1e"
I20260926 11:40:52.284117 14178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4322f45245db4b9e8579d7c98be94ac5" candidate_uuid: "1d08dd932879485aabf14c8b3909a7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac"
I20260926 11:40:52.284204 14458 raft_consensus.cc:3037] T 4322f45245db4b9e8579d7c98be94ac5 P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:52.284215 14178 raft_consensus.cc:3037] T 4322f45245db4b9e8579d7c98be94ac5 P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:52.285173 14178 raft_consensus.cc:2454] T 4322f45245db4b9e8579d7c98be94ac5 P ee1c21f2176e4c1f8f0e41e7702136ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d08dd932879485aabf14c8b3909a7b9 in term 1.
I20260926 11:40:52.285225 14458 raft_consensus.cc:2454] T 4322f45245db4b9e8579d7c98be94ac5 P 039d3bd43cdd4aa382e95b24e2401a1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d08dd932879485aabf14c8b3909a7b9 in term 1.
I20260926 11:40:52.285506 14249 leader_election.cc:302] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039d3bd43cdd4aa382e95b24e2401a1e, 1d08dd932879485aabf14c8b3909a7b9; no voters:
I20260926 11:40:52.285709 14539 raft_consensus.cc:2781] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:52.285905 14539 raft_consensus.cc:686] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9 [term 1 LEADER]: Becoming Leader. State: Replica: 1d08dd932879485aabf14c8b3909a7b9, State: Running, Role: LEADER
I20260926 11:40:52.286204 14539 consensus_queue.cc:237] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } } peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } }
I20260926 11:40:52.289117 14021 catalog_manager.cc:5495] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d08dd932879485aabf14c8b3909a7b9 (127.6.96.2). New cstate: current_term: 1 leader_uuid: "1d08dd932879485aabf14c8b3909a7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } health_report { overall_health: UNKNOWN } } }
I20260926 11:40:52.327618 14528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.324487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55496 (local address 127.6.96.2:42263)
0926 11:40:52.324610 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:52.324613 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:52.324626 (+ 13us) server_negotiation.cc:407] Connection header received
0926 11:40:52.324721 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:52.324723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:52.324781 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:52.324870 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:40:52.325051 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.325538 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.326334 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.326563 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.326643 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.326869 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:40:52.327319 (+ 450us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:40:52.327402 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:52.327455 (+ 53us) server_negotiation.cc:299] Negotiation successful
0926 11:40:52.327505 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 11:40:52.333364 14178 raft_consensus.cc:1261] T 4322f45245db4b9e8579d7c98be94ac5 P ee1c21f2176e4c1f8f0e41e7702136ac [term 1 FOLLOWER]: Refusing update from remote peer 1d08dd932879485aabf14c8b3909a7b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:52.333619 14458 raft_consensus.cc:1261] T 4322f45245db4b9e8579d7c98be94ac5 P 039d3bd43cdd4aa382e95b24e2401a1e [term 1 FOLLOWER]: Refusing update from remote peer 1d08dd932879485aabf14c8b3909a7b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:52.334029 14539 consensus_queue.cc:1035] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:52.334194 14542 consensus_queue.cc:1035] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:52.342024 14546 mvcc.cc:205] Tried to move back new op lower bound from 7333572003150151680 to 7333572002966745088. Current Snapshot: MvccSnapshot[applied={T|T < 7333572003150151680}]
I20260926 11:40:52.342216 14547 mvcc.cc:205] Tried to move back new op lower bound from 7333572003150151680 to 7333572002966745088. Current Snapshot: MvccSnapshot[applied={T|T < 7333572003150151680}]
I20260926 11:40:52.350934 14020 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:52.351118 14020 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:52.353404 14020 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375: Sending DeleteTablet for 3 replicas of tablet 4322f45245db4b9e8579d7c98be94ac5
I20260926 11:40:52.354185 14294 tablet_service.cc:1514] Processing DeleteTablet for tablet 4322f45245db4b9e8579d7c98be94ac5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:52 UTC) from {username='oryx', principal='oryx/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:55486
I20260926 11:40:52.354327 14438 tablet_service.cc:1514] Processing DeleteTablet for tablet 4322f45245db4b9e8579d7c98be94ac5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:52 UTC) from {username='oryx', principal='oryx/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:38152
I20260926 11:40:52.354357 14158 tablet_service.cc:1514] Processing DeleteTablet for tablet 4322f45245db4b9e8579d7c98be94ac5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:52 UTC) from {username='oryx', principal='oryx/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:56996
I20260926 11:40:52.354879 14555 tablet_replica.cc:330] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9: stopping tablet replica
I20260926 11:40:52.355237 14555 raft_consensus.cc:2229] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 11:40:52.355268 14556 tablet_replica.cc:330] T 4322f45245db4b9e8579d7c98be94ac5 P 039d3bd43cdd4aa382e95b24e2401a1e: stopping tablet replica
I20260926 11:40:52.355511 14020 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 11:40:52.355736 14555 raft_consensus.cc:2258] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 11:40:52.355944 14020 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:40:52.356829 14556 raft_consensus.cc:2229] T 4322f45245db4b9e8579d7c98be94ac5 P 039d3bd43cdd4aa382e95b24e2401a1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:52.357196 14556 raft_consensus.cc:2258] T 4322f45245db4b9e8579d7c98be94ac5 P 039d3bd43cdd4aa382e95b24e2401a1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:52.357286 14555 ts_tablet_manager.cc:1905] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:52.360239 14555 ts_tablet_manager.cc:1918] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:40:52.360399 14555 log.cc:1196] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-1/wal/wals/4322f45245db4b9e8579d7c98be94ac5
I20260926 11:40:52.360834 14158 tablet_service.cc:1467] Processing CreateTablet for tablet 39a354f70f374635a834ffa8291a9f9c (DEFAULT_TABLE table=test-table [id=8d674ad1ba974f87b4047140c289b0f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:52.360927 14294 tablet_service.cc:1467] Processing CreateTablet for tablet 39a354f70f374635a834ffa8291a9f9c (DEFAULT_TABLE table=test-table [id=8d674ad1ba974f87b4047140c289b0f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:52.361043 14438 tablet_service.cc:1467] Processing CreateTablet for tablet 39a354f70f374635a834ffa8291a9f9c (DEFAULT_TABLE table=test-table [id=8d674ad1ba974f87b4047140c289b0f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:52.361112 14158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39a354f70f374635a834ffa8291a9f9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:52.361182 14294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39a354f70f374635a834ffa8291a9f9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:52.361253 14438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39a354f70f374635a834ffa8291a9f9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:52.363304 14533 tablet_bootstrap.cc:492] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9: Bootstrap starting.
I20260926 11:40:52.363368 14532 tablet_bootstrap.cc:492] T 39a354f70f374635a834ffa8291a9f9c P 039d3bd43cdd4aa382e95b24e2401a1e: Bootstrap starting.
I20260926 11:40:52.364233 14533 tablet_bootstrap.cc:654] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:52.364274 14532 tablet_bootstrap.cc:654] T 39a354f70f374635a834ffa8291a9f9c P 039d3bd43cdd4aa382e95b24e2401a1e: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:52.365346 14555 ts_tablet_manager.cc:1939] T 4322f45245db4b9e8579d7c98be94ac5 P 1d08dd932879485aabf14c8b3909a7b9: Deleting consensus metadata
I20260926 11:40:52.365391 14556 ts_tablet_manager.cc:1905] T 4322f45245db4b9e8579d7c98be94ac5 P 039d3bd43cdd4aa382e95b24e2401a1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:52.366459 14008 catalog_manager.cc:4841] TS 1d08dd932879485aabf14c8b3909a7b9 (127.6.96.2:42263): tablet 4322f45245db4b9e8579d7c98be94ac5 (table test-table [id=7293e4b9be3e4f7792c6559c92ce0d03]) successfully deleted
I20260926 11:40:52.367331 14532 tablet_bootstrap.cc:492] T 39a354f70f374635a834ffa8291a9f9c P 039d3bd43cdd4aa382e95b24e2401a1e: No bootstrap required, opened a new log
I20260926 11:40:52.367419 14532 ts_tablet_manager.cc:1397] T 39a354f70f374635a834ffa8291a9f9c P 039d3bd43cdd4aa382e95b24e2401a1e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260926 11:40:52.367672 14557 tablet_replica.cc:330] T 4322f45245db4b9e8579d7c98be94ac5 P ee1c21f2176e4c1f8f0e41e7702136ac: stopping tablet replica
I20260926 11:40:52.367904 14532 raft_consensus.cc:348] T 39a354f70f374635a834ffa8291a9f9c P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.368003 14532 raft_consensus.cc:374] T 39a354f70f374635a834ffa8291a9f9c P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:52.368046 14532 raft_consensus.cc:729] T 39a354f70f374635a834ffa8291a9f9c P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039d3bd43cdd4aa382e95b24e2401a1e, State: Initialized, Role: FOLLOWER
I20260926 11:40:52.368089 14557 raft_consensus.cc:2229] T 4322f45245db4b9e8579d7c98be94ac5 P ee1c21f2176e4c1f8f0e41e7702136ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:52.368160 14532 consensus_queue.cc:260] T 39a354f70f374635a834ffa8291a9f9c P 039d3bd43cdd4aa382e95b24e2401a1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.368418 14532 ts_tablet_manager.cc:1428] T 39a354f70f374635a834ffa8291a9f9c P 039d3bd43cdd4aa382e95b24e2401a1e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:52.368444 14557 raft_consensus.cc:2258] T 4322f45245db4b9e8579d7c98be94ac5 P ee1c21f2176e4c1f8f0e41e7702136ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:52.369275 14556 ts_tablet_manager.cc:1918] T 4322f45245db4b9e8579d7c98be94ac5 P 039d3bd43cdd4aa382e95b24e2401a1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:40:52.369349 14556 log.cc:1196] T 4322f45245db4b9e8579d7c98be94ac5 P 039d3bd43cdd4aa382e95b24e2401a1e: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/4322f45245db4b9e8579d7c98be94ac5
I20260926 11:40:52.369867 14556 ts_tablet_manager.cc:1939] T 4322f45245db4b9e8579d7c98be94ac5 P 039d3bd43cdd4aa382e95b24e2401a1e: Deleting consensus metadata
I20260926 11:40:52.370244 14535 tablet_bootstrap.cc:492] T 39a354f70f374635a834ffa8291a9f9c P ee1c21f2176e4c1f8f0e41e7702136ac: Bootstrap starting.
I20260926 11:40:52.370942 14005 catalog_manager.cc:4841] TS 039d3bd43cdd4aa382e95b24e2401a1e (127.6.96.3:37865): tablet 4322f45245db4b9e8579d7c98be94ac5 (table test-table [id=7293e4b9be3e4f7792c6559c92ce0d03]) successfully deleted
I20260926 11:40:52.371200 14535 tablet_bootstrap.cc:654] T 39a354f70f374635a834ffa8291a9f9c P ee1c21f2176e4c1f8f0e41e7702136ac: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:52.371732 14557 ts_tablet_manager.cc:1905] T 4322f45245db4b9e8579d7c98be94ac5 P ee1c21f2176e4c1f8f0e41e7702136ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:52.372294 14535 tablet_bootstrap.cc:492] T 39a354f70f374635a834ffa8291a9f9c P ee1c21f2176e4c1f8f0e41e7702136ac: No bootstrap required, opened a new log
I20260926 11:40:52.372368 14535 ts_tablet_manager.cc:1397] T 39a354f70f374635a834ffa8291a9f9c P ee1c21f2176e4c1f8f0e41e7702136ac: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:52.372756 14535 raft_consensus.cc:348] T 39a354f70f374635a834ffa8291a9f9c P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.372861 14535 raft_consensus.cc:374] T 39a354f70f374635a834ffa8291a9f9c P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:52.372890 14535 raft_consensus.cc:729] T 39a354f70f374635a834ffa8291a9f9c P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee1c21f2176e4c1f8f0e41e7702136ac, State: Initialized, Role: FOLLOWER
I20260926 11:40:52.372999 14535 consensus_queue.cc:260] T 39a354f70f374635a834ffa8291a9f9c P ee1c21f2176e4c1f8f0e41e7702136ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.373242 14535 ts_tablet_manager.cc:1428] T 39a354f70f374635a834ffa8291a9f9c P ee1c21f2176e4c1f8f0e41e7702136ac: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:52.374487 14557 ts_tablet_manager.cc:1918] T 4322f45245db4b9e8579d7c98be94ac5 P ee1c21f2176e4c1f8f0e41e7702136ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:40:52.374575 14557 log.cc:1196] T 4322f45245db4b9e8579d7c98be94ac5 P ee1c21f2176e4c1f8f0e41e7702136ac: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-0/wal/wals/4322f45245db4b9e8579d7c98be94ac5
I20260926 11:40:52.375314 14557 ts_tablet_manager.cc:1939] T 4322f45245db4b9e8579d7c98be94ac5 P ee1c21f2176e4c1f8f0e41e7702136ac: Deleting consensus metadata
I20260926 11:40:52.375438 14533 tablet_bootstrap.cc:492] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9: No bootstrap required, opened a new log
I20260926 11:40:52.375521 14533 ts_tablet_manager.cc:1397] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.002s
I20260926 11:40:52.375955 14533 raft_consensus.cc:348] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.376055 14533 raft_consensus.cc:374] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:52.376082 14533 raft_consensus.cc:729] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d08dd932879485aabf14c8b3909a7b9, State: Initialized, Role: FOLLOWER
I20260926 11:40:52.376194 14533 consensus_queue.cc:260] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.376331 14006 catalog_manager.cc:4841] TS ee1c21f2176e4c1f8f0e41e7702136ac (127.6.96.1:42589): tablet 4322f45245db4b9e8579d7c98be94ac5 (table test-table [id=7293e4b9be3e4f7792c6559c92ce0d03]) successfully deleted
I20260926 11:40:52.376432 14533 ts_tablet_manager.cc:1428] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260926 11:40:52.406243 14365 tablet.cc:2369] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 11:40:52.438386 14225 tablet.cc:2369] T 39a354f70f374635a834ffa8291a9f9c P ee1c21f2176e4c1f8f0e41e7702136ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:40:52.480777 14550 raft_consensus.cc:482] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:40:52.480908 14550 raft_consensus.cc:504] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.481341 14550 leader_election.cc:288] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee1c21f2176e4c1f8f0e41e7702136ac (127.6.96.1:42589), 039d3bd43cdd4aa382e95b24e2401a1e (127.6.96.3:37865)
I20260926 11:40:52.481778 14178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39a354f70f374635a834ffa8291a9f9c" candidate_uuid: "1d08dd932879485aabf14c8b3909a7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" is_pre_election: true
I20260926 11:40:52.481798 14458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39a354f70f374635a834ffa8291a9f9c" candidate_uuid: "1d08dd932879485aabf14c8b3909a7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" is_pre_election: true
I20260926 11:40:52.481932 14178 raft_consensus.cc:2454] T 39a354f70f374635a834ffa8291a9f9c P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d08dd932879485aabf14c8b3909a7b9 in term 0.
I20260926 11:40:52.481932 14458 raft_consensus.cc:2454] T 39a354f70f374635a834ffa8291a9f9c P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d08dd932879485aabf14c8b3909a7b9 in term 0.
I20260926 11:40:52.482219 14250 leader_election.cc:302] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d08dd932879485aabf14c8b3909a7b9, ee1c21f2176e4c1f8f0e41e7702136ac; no voters:
I20260926 11:40:52.482410 14550 raft_consensus.cc:2781] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:40:52.482488 14550 raft_consensus.cc:482] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:40:52.482522 14550 raft_consensus.cc:3037] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:52.483829 14550 raft_consensus.cc:504] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.484191 14550 leader_election.cc:288] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9 [CANDIDATE]: Term 1 election: Requested vote from peers ee1c21f2176e4c1f8f0e41e7702136ac (127.6.96.1:42589), 039d3bd43cdd4aa382e95b24e2401a1e (127.6.96.3:37865)
I20260926 11:40:52.484526 14458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39a354f70f374635a834ffa8291a9f9c" candidate_uuid: "1d08dd932879485aabf14c8b3909a7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039d3bd43cdd4aa382e95b24e2401a1e"
I20260926 11:40:52.484656 14458 raft_consensus.cc:3037] T 39a354f70f374635a834ffa8291a9f9c P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:52.484622 14178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39a354f70f374635a834ffa8291a9f9c" candidate_uuid: "1d08dd932879485aabf14c8b3909a7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac"
I20260926 11:40:52.484740 14178 raft_consensus.cc:3037] T 39a354f70f374635a834ffa8291a9f9c P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:52.485698 14458 raft_consensus.cc:2454] T 39a354f70f374635a834ffa8291a9f9c P 039d3bd43cdd4aa382e95b24e2401a1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d08dd932879485aabf14c8b3909a7b9 in term 1.
I20260926 11:40:52.485767 14178 raft_consensus.cc:2454] T 39a354f70f374635a834ffa8291a9f9c P ee1c21f2176e4c1f8f0e41e7702136ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d08dd932879485aabf14c8b3909a7b9 in term 1.
I20260926 11:40:52.486047 14250 leader_election.cc:302] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d08dd932879485aabf14c8b3909a7b9, ee1c21f2176e4c1f8f0e41e7702136ac; no voters:
I20260926 11:40:52.486202 14550 raft_consensus.cc:2781] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:52.486292 14550 raft_consensus.cc:686] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9 [term 1 LEADER]: Becoming Leader. State: Replica: 1d08dd932879485aabf14c8b3909a7b9, State: Running, Role: LEADER
I20260926 11:40:52.486421 14550 consensus_queue.cc:237] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.487931 14020 catalog_manager.cc:5495] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d08dd932879485aabf14c8b3909a7b9 (127.6.96.2). New cstate: current_term: 1 leader_uuid: "1d08dd932879485aabf14c8b3909a7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } health_report { overall_health: UNKNOWN } } }
Sep 26 11:40:52 dist-test-slave-xxxx krb5kdc[13976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422851, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.62@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM
I20260926 11:40:52.540681 14070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.532653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54656 (local address 127.6.96.62:39759)
0926 11:40:52.532783 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:52.532788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:52.532949 (+ 161us) server_negotiation.cc:407] Connection header received
0926 11:40:52.533072 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:52.533075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:52.533114 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:52.533176 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:52.533846 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.534350 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.535157 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.535357 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.535441 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.537743 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:52.537747 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:52.537748 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:52.537769 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:52.539106 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:52.539528 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:52.539531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:52.539533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:52.539605 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:52.539828 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:52.539831 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:52.539833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:52.540003 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:52.540086 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:52.540370 (+ 284us) server_negotiation.cc:299] Negotiation successful
0926 11:40:52.540524 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":52}
I20260926 11:40:52.540681 14567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.532598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:39759 (local address 127.0.0.1:54656)
0926 11:40:52.532865 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:52.532877 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:52.532990 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:52.533203 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:52.533206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:52.533628 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:52.533729 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:52.533736 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.534448 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.534451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:52.535042 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:52.535051 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.535452 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.535455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:52.535570 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.536107 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:52.536119 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:52.537599 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:52.539203 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:52.539213 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:52.539219 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:52.539441 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:52.539679 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:52.539682 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:52.539684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:52.539757 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:52.540096 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:52.540104 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:52.540218 (+ 114us) client_negotiation.cc:769] Sending connection context
0926 11:40:52.540312 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 11:40:52.540475 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":30,"thread_start_us":105,"threads_started":1}
I20260926 11:40:52.544576 14020 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:54656:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 11:40:52.545298 14020 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:40:52.549880 14158 tablet_service.cc:1467] Processing CreateTablet for tablet 0f8102e702344f08ae1812a3e4a2d958 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e14a5b56948946b4a55c5978928580a7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 11:40:52.549950 14294 tablet_service.cc:1467] Processing CreateTablet for tablet 0f8102e702344f08ae1812a3e4a2d958 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e14a5b56948946b4a55c5978928580a7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 11:40:52.550019 14438 tablet_service.cc:1467] Processing CreateTablet for tablet 0f8102e702344f08ae1812a3e4a2d958 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e14a5b56948946b4a55c5978928580a7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 11:40:52.550253 14158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f8102e702344f08ae1812a3e4a2d958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:52.550324 14294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f8102e702344f08ae1812a3e4a2d958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:52.550323 14438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f8102e702344f08ae1812a3e4a2d958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:52.552831 14535 tablet_bootstrap.cc:492] T 0f8102e702344f08ae1812a3e4a2d958 P ee1c21f2176e4c1f8f0e41e7702136ac: Bootstrap starting.
I20260926 11:40:52.553012 14533 tablet_bootstrap.cc:492] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9: Bootstrap starting.
I20260926 11:40:52.553341 14532 tablet_bootstrap.cc:492] T 0f8102e702344f08ae1812a3e4a2d958 P 039d3bd43cdd4aa382e95b24e2401a1e: Bootstrap starting.
I20260926 11:40:52.553761 14535 tablet_bootstrap.cc:654] T 0f8102e702344f08ae1812a3e4a2d958 P ee1c21f2176e4c1f8f0e41e7702136ac: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:52.553761 14533 tablet_bootstrap.cc:654] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:52.554085 14532 tablet_bootstrap.cc:654] T 0f8102e702344f08ae1812a3e4a2d958 P 039d3bd43cdd4aa382e95b24e2401a1e: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:52.554894 14533 tablet_bootstrap.cc:492] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9: No bootstrap required, opened a new log
I20260926 11:40:52.554975 14533 ts_tablet_manager.cc:1397] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 11:40:52.555090 14532 tablet_bootstrap.cc:492] T 0f8102e702344f08ae1812a3e4a2d958 P 039d3bd43cdd4aa382e95b24e2401a1e: No bootstrap required, opened a new log
I20260926 11:40:52.555159 14532 ts_tablet_manager.cc:1397] T 0f8102e702344f08ae1812a3e4a2d958 P 039d3bd43cdd4aa382e95b24e2401a1e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 11:40:52.555339 14533 raft_consensus.cc:348] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.555433 14533 raft_consensus.cc:374] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:52.555461 14533 raft_consensus.cc:729] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d08dd932879485aabf14c8b3909a7b9, State: Initialized, Role: FOLLOWER
I20260926 11:40:52.555507 14532 raft_consensus.cc:348] T 0f8102e702344f08ae1812a3e4a2d958 P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.555575 14533 consensus_queue.cc:260] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.555634 14532 raft_consensus.cc:374] T 0f8102e702344f08ae1812a3e4a2d958 P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:52.555662 14532 raft_consensus.cc:729] T 0f8102e702344f08ae1812a3e4a2d958 P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039d3bd43cdd4aa382e95b24e2401a1e, State: Initialized, Role: FOLLOWER
I20260926 11:40:52.555776 14532 consensus_queue.cc:260] T 0f8102e702344f08ae1812a3e4a2d958 P 039d3bd43cdd4aa382e95b24e2401a1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.555820 14533 ts_tablet_manager.cc:1428] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 11:40:52.555781 14550 tablet_replica.cc:439] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } } }
I20260926 11:40:52.555896 14550 tablet_replica.cc:442] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 11:40:52.556003 14532 ts_tablet_manager.cc:1428] T 0f8102e702344f08ae1812a3e4a2d958 P 039d3bd43cdd4aa382e95b24e2401a1e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:52.556200 14538 tablet_replica.cc:439] T 0f8102e702344f08ae1812a3e4a2d958 P 039d3bd43cdd4aa382e95b24e2401a1e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } } }
I20260926 11:40:52.556293 14538 tablet_replica.cc:442] T 0f8102e702344f08ae1812a3e4a2d958 P 039d3bd43cdd4aa382e95b24e2401a1e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 11:40:52.556281 14535 tablet_bootstrap.cc:492] T 0f8102e702344f08ae1812a3e4a2d958 P ee1c21f2176e4c1f8f0e41e7702136ac: No bootstrap required, opened a new log
I20260926 11:40:52.556350 14535 ts_tablet_manager.cc:1397] T 0f8102e702344f08ae1812a3e4a2d958 P ee1c21f2176e4c1f8f0e41e7702136ac: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260926 11:40:52.556747 14535 raft_consensus.cc:348] T 0f8102e702344f08ae1812a3e4a2d958 P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.556860 14535 raft_consensus.cc:374] T 0f8102e702344f08ae1812a3e4a2d958 P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:52.556900 14535 raft_consensus.cc:729] T 0f8102e702344f08ae1812a3e4a2d958 P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee1c21f2176e4c1f8f0e41e7702136ac, State: Initialized, Role: FOLLOWER
I20260926 11:40:52.557019 14535 consensus_queue.cc:260] T 0f8102e702344f08ae1812a3e4a2d958 P ee1c21f2176e4c1f8f0e41e7702136ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.557261 14535 ts_tablet_manager.cc:1428] T 0f8102e702344f08ae1812a3e4a2d958 P ee1c21f2176e4c1f8f0e41e7702136ac: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:52.557442 14540 tablet_replica.cc:439] T 0f8102e702344f08ae1812a3e4a2d958 P ee1c21f2176e4c1f8f0e41e7702136ac: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } } }
I20260926 11:40:52.557561 14540 tablet_replica.cc:442] T 0f8102e702344f08ae1812a3e4a2d958 P ee1c21f2176e4c1f8f0e41e7702136ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 11:40:52.599053 14550 raft_consensus.cc:482] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:40:52.599177 14550 raft_consensus.cc:504] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.599573 14550 leader_election.cc:288] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee1c21f2176e4c1f8f0e41e7702136ac (127.6.96.1:42589), 039d3bd43cdd4aa382e95b24e2401a1e (127.6.96.3:37865)
I20260926 11:40:52.599987 14178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f8102e702344f08ae1812a3e4a2d958" candidate_uuid: "1d08dd932879485aabf14c8b3909a7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" is_pre_election: true
I20260926 11:40:52.599987 14458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f8102e702344f08ae1812a3e4a2d958" candidate_uuid: "1d08dd932879485aabf14c8b3909a7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" is_pre_election: true
I20260926 11:40:52.600142 14458 raft_consensus.cc:2454] T 0f8102e702344f08ae1812a3e4a2d958 P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d08dd932879485aabf14c8b3909a7b9 in term 0.
I20260926 11:40:52.600142 14178 raft_consensus.cc:2454] T 0f8102e702344f08ae1812a3e4a2d958 P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d08dd932879485aabf14c8b3909a7b9 in term 0.
I20260926 11:40:52.600430 14249 leader_election.cc:302] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 039d3bd43cdd4aa382e95b24e2401a1e, 1d08dd932879485aabf14c8b3909a7b9, ee1c21f2176e4c1f8f0e41e7702136ac; no voters:
I20260926 11:40:52.600646 14550 raft_consensus.cc:2781] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:40:52.600706 14550 raft_consensus.cc:482] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:40:52.600733 14550 raft_consensus.cc:3037] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:52.601660 14550 raft_consensus.cc:504] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.601958 14550 leader_election.cc:288] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9 [CANDIDATE]: Term 1 election: Requested vote from peers ee1c21f2176e4c1f8f0e41e7702136ac (127.6.96.1:42589), 039d3bd43cdd4aa382e95b24e2401a1e (127.6.96.3:37865)
I20260926 11:40:52.602294 14178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f8102e702344f08ae1812a3e4a2d958" candidate_uuid: "1d08dd932879485aabf14c8b3909a7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac"
I20260926 11:40:52.602396 14178 raft_consensus.cc:3037] T 0f8102e702344f08ae1812a3e4a2d958 P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:52.603281 14178 raft_consensus.cc:2454] T 0f8102e702344f08ae1812a3e4a2d958 P ee1c21f2176e4c1f8f0e41e7702136ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d08dd932879485aabf14c8b3909a7b9 in term 1.
I20260926 11:40:52.603545 14250 leader_election.cc:302] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d08dd932879485aabf14c8b3909a7b9, ee1c21f2176e4c1f8f0e41e7702136ac; no voters:
I20260926 11:40:52.603523 14458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f8102e702344f08ae1812a3e4a2d958" candidate_uuid: "1d08dd932879485aabf14c8b3909a7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039d3bd43cdd4aa382e95b24e2401a1e"
I20260926 11:40:52.603662 14458 raft_consensus.cc:3037] T 0f8102e702344f08ae1812a3e4a2d958 P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:52.603725 14550 raft_consensus.cc:2781] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:52.603840 14550 raft_consensus.cc:686] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9 [term 1 LEADER]: Becoming Leader. State: Replica: 1d08dd932879485aabf14c8b3909a7b9, State: Running, Role: LEADER
I20260926 11:40:52.603971 14550 consensus_queue.cc:237] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.604598 14539 tablet_replica.cc:439] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9: TxnStatusTablet state changed. Reason: New leader 1d08dd932879485aabf14c8b3909a7b9. Latest consensus state: current_term: 1 leader_uuid: "1d08dd932879485aabf14c8b3909a7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } } }
I20260926 11:40:52.604662 14458 raft_consensus.cc:2454] T 0f8102e702344f08ae1812a3e4a2d958 P 039d3bd43cdd4aa382e95b24e2401a1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d08dd932879485aabf14c8b3909a7b9 in term 1.
I20260926 11:40:52.604717 14539 tablet_replica.cc:442] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9: This TxnStatusTablet replica's current role is: LEADER
I20260926 11:40:52.604909 14572 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 11:40:52.605067 14572 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 11:40:52.605773 14021 catalog_manager.cc:5495] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d08dd932879485aabf14c8b3909a7b9 (127.6.96.2). New cstate: current_term: 1 leader_uuid: "1d08dd932879485aabf14c8b3909a7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } health_report { overall_health: UNKNOWN } } }
W20260926 11:40:52.608995 14505 tablet.cc:2369] T 0f8102e702344f08ae1812a3e4a2d958 P 039d3bd43cdd4aa382e95b24e2401a1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:40:52.625262 14567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.621822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:42263 (local address 127.0.0.1:55500)
0926 11:40:52.621961 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:52.621989 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:52.622074 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:52.622381 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:52.622384 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:52.622395 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 11:40:52.622524 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:52.622529 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.623522 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.623525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:52.624154 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:52.624163 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.624686 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.624688 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:52.624749 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.624793 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 11:40:52.625040 (+ 247us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 11:40:52.625047 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 11:40:52.625090 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 11:40:52.625156 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260926 11:40:52.625283 14528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.621898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55500 (local address 127.6.96.2:42263)
0926 11:40:52.622058 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:52.622063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:52.622079 (+ 16us) server_negotiation.cc:407] Connection header received
0926 11:40:52.622191 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:52.622194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:52.622246 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:52.622324 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:40:52.622614 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.623417 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.624252 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.624598 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.624693 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.624899 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:40:52.624978 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:40:52.625040 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:52.625102 (+ 62us) server_negotiation.cc:299] Negotiation successful
0926 11:40:52.625151 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 11:40:52.627578 14458 raft_consensus.cc:1261] T 0f8102e702344f08ae1812a3e4a2d958 P 039d3bd43cdd4aa382e95b24e2401a1e [term 1 FOLLOWER]: Refusing update from remote peer 1d08dd932879485aabf14c8b3909a7b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:52.627620 14178 raft_consensus.cc:1261] T 0f8102e702344f08ae1812a3e4a2d958 P ee1c21f2176e4c1f8f0e41e7702136ac [term 1 FOLLOWER]: Refusing update from remote peer 1d08dd932879485aabf14c8b3909a7b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:52.627983 14539 consensus_queue.cc:1035] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:52.628144 14550 consensus_queue.cc:1035] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:52.629307 14538 tablet_replica.cc:439] T 0f8102e702344f08ae1812a3e4a2d958 P 039d3bd43cdd4aa382e95b24e2401a1e: TxnStatusTablet state changed. Reason: New leader 1d08dd932879485aabf14c8b3909a7b9. Latest consensus state: current_term: 1 leader_uuid: "1d08dd932879485aabf14c8b3909a7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } } }
I20260926 11:40:52.629410 14538 tablet_replica.cc:442] T 0f8102e702344f08ae1812a3e4a2d958 P 039d3bd43cdd4aa382e95b24e2401a1e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 11:40:52.630300 14540 tablet_replica.cc:439] T 0f8102e702344f08ae1812a3e4a2d958 P ee1c21f2176e4c1f8f0e41e7702136ac: TxnStatusTablet state changed. Reason: New leader 1d08dd932879485aabf14c8b3909a7b9. Latest consensus state: current_term: 1 leader_uuid: "1d08dd932879485aabf14c8b3909a7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } } }
I20260926 11:40:52.630426 14540 tablet_replica.cc:442] T 0f8102e702344f08ae1812a3e4a2d958 P ee1c21f2176e4c1f8f0e41e7702136ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 11:40:52.631666 14539 tablet_replica.cc:439] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1d08dd932879485aabf14c8b3909a7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } } }
I20260926 11:40:52.631767 14539 tablet_replica.cc:442] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9: This TxnStatusTablet replica's current role is: LEADER
I20260926 11:40:52.631949 14542 tablet_replica.cc:439] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1d08dd932879485aabf14c8b3909a7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } } }
I20260926 11:40:52.632035 14542 tablet_replica.cc:442] T 0f8102e702344f08ae1812a3e4a2d958 P 1d08dd932879485aabf14c8b3909a7b9: This TxnStatusTablet replica's current role is: LEADER
I20260926 11:40:52.632790 14540 tablet_replica.cc:439] T 0f8102e702344f08ae1812a3e4a2d958 P ee1c21f2176e4c1f8f0e41e7702136ac: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1d08dd932879485aabf14c8b3909a7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } } }
I20260926 11:40:52.632889 14540 tablet_replica.cc:442] T 0f8102e702344f08ae1812a3e4a2d958 P ee1c21f2176e4c1f8f0e41e7702136ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 11:40:52.635038 14545 mvcc.cc:205] Tried to move back new op lower bound from 7333572004358631424 to 7333572004267212800. Current Snapshot: MvccSnapshot[applied={T|T < 7333572004358631424 or (T in {7333572004358631424})}]
I20260926 11:40:52.636329 14538 tablet_replica.cc:439] T 0f8102e702344f08ae1812a3e4a2d958 P 039d3bd43cdd4aa382e95b24e2401a1e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1d08dd932879485aabf14c8b3909a7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } } }
I20260926 11:40:52.636448 14538 tablet_replica.cc:442] T 0f8102e702344f08ae1812a3e4a2d958 P 039d3bd43cdd4aa382e95b24e2401a1e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 11:40:52.645692 14528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.642681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:44345 (local address 127.6.96.2:42263)
0926 11:40:52.642785 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:52.642790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:52.642983 (+ 193us) server_negotiation.cc:407] Connection header received
0926 11:40:52.643088 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:52.643091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:52.643141 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:52.643197 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:40:52.643419 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.643914 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.644711 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.644916 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.644997 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.645290 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:40:52.645359 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:40:52.645412 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:52.645495 (+ 83us) server_negotiation.cc:299] Negotiation successful
0926 11:40:52.645546 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":33}
I20260926 11:40:52.645692 14579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.642625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:42263 (local address 127.6.96.2:44345)
0926 11:40:52.642913 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:52.642925 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:52.643019 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:52.643214 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:52.643217 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:52.643224 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 11:40:52.643336 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:52.643341 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.644008 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.644011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:52.644607 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:52.644622 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.645010 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.645013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:52.645139 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.645181 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 11:40:52.645427 (+ 246us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 11:40:52.645433 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 11:40:52.645490 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 11:40:52.645572 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":41,"thread_start_us":105,"threads_started":1}
I20260926 11:40:52.652320 14178 raft_consensus.cc:1261] T 39a354f70f374635a834ffa8291a9f9c P ee1c21f2176e4c1f8f0e41e7702136ac [term 1 FOLLOWER]: Refusing update from remote peer 1d08dd932879485aabf14c8b3909a7b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:52.652343 14458 raft_consensus.cc:1261] T 39a354f70f374635a834ffa8291a9f9c P 039d3bd43cdd4aa382e95b24e2401a1e [term 1 FOLLOWER]: Refusing update from remote peer 1d08dd932879485aabf14c8b3909a7b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:52.652726 14542 consensus_queue.cc:1035] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:52.652894 14539 consensus_queue.cc:1035] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:52.687608 14017 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:52.687752 14017 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:52.689745 14017 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375: Sending DeleteTablet for 3 replicas of tablet 39a354f70f374635a834ffa8291a9f9c
I20260926 11:40:52.690238 14158 tablet_service.cc:1514] Processing DeleteTablet for tablet 39a354f70f374635a834ffa8291a9f9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:52 UTC) from {username='oryx', principal='oryx/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:56996
I20260926 11:40:52.690354 14438 tablet_service.cc:1514] Processing DeleteTablet for tablet 39a354f70f374635a834ffa8291a9f9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:52 UTC) from {username='oryx', principal='oryx/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:38152
I20260926 11:40:52.690433 14557 tablet_replica.cc:330] T 39a354f70f374635a834ffa8291a9f9c P ee1c21f2176e4c1f8f0e41e7702136ac: stopping tablet replica
I20260926 11:40:52.690562 14557 raft_consensus.cc:2229] T 39a354f70f374635a834ffa8291a9f9c P ee1c21f2176e4c1f8f0e41e7702136ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:52.690670 14557 raft_consensus.cc:2258] T 39a354f70f374635a834ffa8291a9f9c P ee1c21f2176e4c1f8f0e41e7702136ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:52.690742 14298 tablet_service.cc:1514] Processing DeleteTablet for tablet 39a354f70f374635a834ffa8291a9f9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:52 UTC) from {username='oryx', principal='oryx/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:55486
I20260926 11:40:52.690821 14556 tablet_replica.cc:330] T 39a354f70f374635a834ffa8291a9f9c P 039d3bd43cdd4aa382e95b24e2401a1e: stopping tablet replica
I20260926 11:40:52.690927 14555 tablet_replica.cc:330] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9: stopping tablet replica
I20260926 11:40:52.690968 14556 raft_consensus.cc:2229] T 39a354f70f374635a834ffa8291a9f9c P 039d3bd43cdd4aa382e95b24e2401a1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:52.691033 14555 raft_consensus.cc:2229] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 11:40:52.691064 14556 raft_consensus.cc:2258] T 39a354f70f374635a834ffa8291a9f9c P 039d3bd43cdd4aa382e95b24e2401a1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:52.691197 14555 raft_consensus.cc:2258] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:52.691895 14555 ts_tablet_manager.cc:1905] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:52.691982 14557 ts_tablet_manager.cc:1905] T 39a354f70f374635a834ffa8291a9f9c P ee1c21f2176e4c1f8f0e41e7702136ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:52.694626 14555 ts_tablet_manager.cc:1918] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 11:40:52.694693 14555 log.cc:1196] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-1/wal/wals/39a354f70f374635a834ffa8291a9f9c
I20260926 11:40:52.694674 14557 ts_tablet_manager.cc:1918] T 39a354f70f374635a834ffa8291a9f9c P ee1c21f2176e4c1f8f0e41e7702136ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 11:40:52.694743 14557 log.cc:1196] T 39a354f70f374635a834ffa8291a9f9c P ee1c21f2176e4c1f8f0e41e7702136ac: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-0/wal/wals/39a354f70f374635a834ffa8291a9f9c
I20260926 11:40:52.694922 14556 ts_tablet_manager.cc:1905] T 39a354f70f374635a834ffa8291a9f9c P 039d3bd43cdd4aa382e95b24e2401a1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:52.694959 14555 ts_tablet_manager.cc:1939] T 39a354f70f374635a834ffa8291a9f9c P 1d08dd932879485aabf14c8b3909a7b9: Deleting consensus metadata
I20260926 11:40:52.695073 14557 ts_tablet_manager.cc:1939] T 39a354f70f374635a834ffa8291a9f9c P ee1c21f2176e4c1f8f0e41e7702136ac: Deleting consensus metadata
I20260926 11:40:52.695887 14008 catalog_manager.cc:4841] TS 1d08dd932879485aabf14c8b3909a7b9 (127.6.96.2:42263): tablet 39a354f70f374635a834ffa8291a9f9c (table test-table [id=8d674ad1ba974f87b4047140c289b0f1]) successfully deleted
I20260926 11:40:52.697264 14556 ts_tablet_manager.cc:1918] T 39a354f70f374635a834ffa8291a9f9c P 039d3bd43cdd4aa382e95b24e2401a1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 11:40:52.697327 14556 log.cc:1196] T 39a354f70f374635a834ffa8291a9f9c P 039d3bd43cdd4aa382e95b24e2401a1e: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/39a354f70f374635a834ffa8291a9f9c
I20260926 11:40:52.697533 14556 ts_tablet_manager.cc:1939] T 39a354f70f374635a834ffa8291a9f9c P 039d3bd43cdd4aa382e95b24e2401a1e: Deleting consensus metadata
I20260926 11:40:52.698181 14005 catalog_manager.cc:4841] TS 039d3bd43cdd4aa382e95b24e2401a1e (127.6.96.3:37865): tablet 39a354f70f374635a834ffa8291a9f9c (table test-table [id=8d674ad1ba974f87b4047140c289b0f1]) successfully deleted
I20260926 11:40:52.698364 14006 catalog_manager.cc:4841] TS ee1c21f2176e4c1f8f0e41e7702136ac (127.6.96.1:42589): tablet 39a354f70f374635a834ffa8291a9f9c (table test-table [id=8d674ad1ba974f87b4047140c289b0f1]) successfully deleted
Sep 26 11:40:52 dist-test-slave-xxxx krb5kdc[13976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 11:40:52 dist-test-slave-xxxx krb5kdc[13976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:40:52 dist-test-slave-xxxx krb5kdc[13976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
W20260926 11:40:52.720206 14070 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:52.713916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54660 (local address 127.6.96.62:39759)
0926 11:40:52.714049 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:52.714054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:52.714228 (+ 174us) server_negotiation.cc:407] Connection header received
0926 11:40:52.714343 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:52.714346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:52.714391 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:52.714452 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:52.715192 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.715711 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.716529 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.716748 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.716844 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.720087 (+ 3243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54660: BlockingRecv error: recv got EOF from 127.0.0.1:54660 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 26 11:40:52 dist-test-slave-xxxx krb5kdc[13976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM
I20260926 11:40:52.730916 14070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.721851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54664 (local address 127.6.96.62:39759)
0926 11:40:52.721949 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:52.721953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:52.722290 (+ 337us) server_negotiation.cc:407] Connection header received
0926 11:40:52.722652 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:52.722656 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:52.722705 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:52.722755 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:52.723528 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.724024 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.725107 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.725470 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.725541 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.728094 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:52.728096 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:52.728098 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:52.728133 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:52.729343 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:52.729800 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:52.729803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:52.729804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:52.729867 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:52.730130 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:52.730132 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:52.730133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:52.730285 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:52.730374 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:52.730644 (+ 270us) server_negotiation.cc:299] Negotiation successful
0926 11:40:52.730731 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 11:40:52.733372 14017 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 11:40:52.733784 14017 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:40:52.737977 14158 tablet_service.cc:1467] Processing CreateTablet for tablet d6cff98a906f41479dbee0d3ab8073dd (DEFAULT_TABLE table=test-table [id=7f30ffd43d8d4c6cbc50567a19c09a80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:52.738127 14438 tablet_service.cc:1467] Processing CreateTablet for tablet d6cff98a906f41479dbee0d3ab8073dd (DEFAULT_TABLE table=test-table [id=7f30ffd43d8d4c6cbc50567a19c09a80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:52.738308 14158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6cff98a906f41479dbee0d3ab8073dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:52.738360 14438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6cff98a906f41479dbee0d3ab8073dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:52.740604 14532 tablet_bootstrap.cc:492] T d6cff98a906f41479dbee0d3ab8073dd P 039d3bd43cdd4aa382e95b24e2401a1e: Bootstrap starting.
I20260926 11:40:52.740795 14298 tablet_service.cc:1467] Processing CreateTablet for tablet d6cff98a906f41479dbee0d3ab8073dd (DEFAULT_TABLE table=test-table [id=7f30ffd43d8d4c6cbc50567a19c09a80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:52.741066 14298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6cff98a906f41479dbee0d3ab8073dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:52.741555 14532 tablet_bootstrap.cc:654] T d6cff98a906f41479dbee0d3ab8073dd P 039d3bd43cdd4aa382e95b24e2401a1e: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:52.742125 14535 tablet_bootstrap.cc:492] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac: Bootstrap starting.
I20260926 11:40:52.742990 14535 tablet_bootstrap.cc:654] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:52.743053 14533 tablet_bootstrap.cc:492] T d6cff98a906f41479dbee0d3ab8073dd P 1d08dd932879485aabf14c8b3909a7b9: Bootstrap starting.
I20260926 11:40:52.743801 14532 tablet_bootstrap.cc:492] T d6cff98a906f41479dbee0d3ab8073dd P 039d3bd43cdd4aa382e95b24e2401a1e: No bootstrap required, opened a new log
I20260926 11:40:52.743878 14532 ts_tablet_manager.cc:1397] T d6cff98a906f41479dbee0d3ab8073dd P 039d3bd43cdd4aa382e95b24e2401a1e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 11:40:52.743984 14533 tablet_bootstrap.cc:654] T d6cff98a906f41479dbee0d3ab8073dd P 1d08dd932879485aabf14c8b3909a7b9: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:52.744266 14532 raft_consensus.cc:348] T d6cff98a906f41479dbee0d3ab8073dd P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.744362 14532 raft_consensus.cc:374] T d6cff98a906f41479dbee0d3ab8073dd P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:52.744387 14532 raft_consensus.cc:729] T d6cff98a906f41479dbee0d3ab8073dd P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039d3bd43cdd4aa382e95b24e2401a1e, State: Initialized, Role: FOLLOWER
I20260926 11:40:52.744505 14532 consensus_queue.cc:260] T d6cff98a906f41479dbee0d3ab8073dd P 039d3bd43cdd4aa382e95b24e2401a1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.744731 14532 ts_tablet_manager.cc:1428] T d6cff98a906f41479dbee0d3ab8073dd P 039d3bd43cdd4aa382e95b24e2401a1e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:52.745079 14533 tablet_bootstrap.cc:492] T d6cff98a906f41479dbee0d3ab8073dd P 1d08dd932879485aabf14c8b3909a7b9: No bootstrap required, opened a new log
I20260926 11:40:52.745178 14533 ts_tablet_manager.cc:1397] T d6cff98a906f41479dbee0d3ab8073dd P 1d08dd932879485aabf14c8b3909a7b9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:52.745589 14533 raft_consensus.cc:348] T d6cff98a906f41479dbee0d3ab8073dd P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.745692 14533 raft_consensus.cc:374] T d6cff98a906f41479dbee0d3ab8073dd P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:52.745721 14533 raft_consensus.cc:729] T d6cff98a906f41479dbee0d3ab8073dd P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d08dd932879485aabf14c8b3909a7b9, State: Initialized, Role: FOLLOWER
I20260926 11:40:52.745808 14535 tablet_bootstrap.cc:492] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac: No bootstrap required, opened a new log
I20260926 11:40:52.745874 14535 ts_tablet_manager.cc:1397] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260926 11:40:52.745836 14533 consensus_queue.cc:260] T d6cff98a906f41479dbee0d3ab8073dd P 1d08dd932879485aabf14c8b3909a7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.746071 14533 ts_tablet_manager.cc:1428] T d6cff98a906f41479dbee0d3ab8073dd P 1d08dd932879485aabf14c8b3909a7b9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:52.746241 14535 raft_consensus.cc:348] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.746332 14535 raft_consensus.cc:374] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:52.746356 14535 raft_consensus.cc:729] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee1c21f2176e4c1f8f0e41e7702136ac, State: Initialized, Role: FOLLOWER
I20260926 11:40:52.746467 14535 consensus_queue.cc:260] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.746687 14535 ts_tablet_manager.cc:1428] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:52.769330 14540 raft_consensus.cc:482] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:40:52.769456 14540 raft_consensus.cc:504] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.770512 14540 leader_election.cc:288] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d08dd932879485aabf14c8b3909a7b9 (127.6.96.2:42263), 039d3bd43cdd4aa382e95b24e2401a1e (127.6.96.3:37865)
I20260926 11:40:52.774436 14528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.771048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:48213 (local address 127.6.96.2:42263)
0926 11:40:52.771166 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:52.771170 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:52.771211 (+ 41us) server_negotiation.cc:407] Connection header received
0926 11:40:52.771348 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:52.771351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:52.771392 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:52.771459 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:52.772057 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.772583 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.773535 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.773909 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.774001 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.774061 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:52.774260 (+ 199us) server_negotiation.cc:299] Negotiation successful
0926 11:40:52.774328 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 11:40:52.774464 14599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.770753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:42263 (local address 127.6.96.1:48213)
0926 11:40:52.771143 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:52.771155 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:52.771253 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:52.771486 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:52.771488 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:52.771506 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:52.771931 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:52.771938 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.772684 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.772687 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:52.773426 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:52.773436 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.774031 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.774035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:52.774122 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.774171 (+ 49us) client_negotiation.cc:769] Sending connection context
0926 11:40:52.774235 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 11:40:52.774309 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":48,"thread_start_us":104,"threads_started":1}
I20260926 11:40:52.774437 14530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.770811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:37817 (local address 127.6.96.3:37865)
0926 11:40:52.770956 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:52.770961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:52.771101 (+ 140us) server_negotiation.cc:407] Connection header received
0926 11:40:52.771249 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:52.771253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:52.771296 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:52.771342 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:40:52.771756 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.772349 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.773509 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.773891 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.773971 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.774033 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:52.774250 (+ 217us) server_negotiation.cc:299] Negotiation successful
0926 11:40:52.774319 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 11:40:52.774471 14598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.770753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:37865 (local address 127.6.96.1:37817)
0926 11:40:52.771000 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:52.771014 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:52.771122 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:52.771474 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:52.771477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:52.771493 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:40:52.771642 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:52.771648 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.772473 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.772476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:52.773363 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:52.773371 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.774019 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.774023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:52.774121 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.774164 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 11:40:52.774223 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 11:40:52.774297 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":5,"spinlock_wait_cycles":1280,"thread_start_us":90,"threads_started":1}
I20260926 11:40:52.775053 14458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6cff98a906f41479dbee0d3ab8073dd" candidate_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" is_pre_election: true
I20260926 11:40:52.775049 14318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6cff98a906f41479dbee0d3ab8073dd" candidate_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d08dd932879485aabf14c8b3909a7b9" is_pre_election: true
I20260926 11:40:52.775175 14458 raft_consensus.cc:2454] T d6cff98a906f41479dbee0d3ab8073dd P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee1c21f2176e4c1f8f0e41e7702136ac in term 0.
I20260926 11:40:52.775262 14318 raft_consensus.cc:2454] T d6cff98a906f41479dbee0d3ab8073dd P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee1c21f2176e4c1f8f0e41e7702136ac in term 0.
I20260926 11:40:52.775502 14109 leader_election.cc:302] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039d3bd43cdd4aa382e95b24e2401a1e, ee1c21f2176e4c1f8f0e41e7702136ac; no voters:
I20260926 11:40:52.775738 14540 raft_consensus.cc:2781] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:40:52.775787 14540 raft_consensus.cc:482] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:40:52.775830 14540 raft_consensus.cc:3037] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:52.776734 14540 raft_consensus.cc:504] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.777024 14540 leader_election.cc:288] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac [CANDIDATE]: Term 1 election: Requested vote from peers 1d08dd932879485aabf14c8b3909a7b9 (127.6.96.2:42263), 039d3bd43cdd4aa382e95b24e2401a1e (127.6.96.3:37865)
I20260926 11:40:52.777309 14318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6cff98a906f41479dbee0d3ab8073dd" candidate_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d08dd932879485aabf14c8b3909a7b9"
I20260926 11:40:52.777334 14458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6cff98a906f41479dbee0d3ab8073dd" candidate_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039d3bd43cdd4aa382e95b24e2401a1e"
I20260926 11:40:52.777408 14318 raft_consensus.cc:3037] T d6cff98a906f41479dbee0d3ab8073dd P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:52.777436 14458 raft_consensus.cc:3037] T d6cff98a906f41479dbee0d3ab8073dd P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:52.778244 14318 raft_consensus.cc:2454] T d6cff98a906f41479dbee0d3ab8073dd P 1d08dd932879485aabf14c8b3909a7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee1c21f2176e4c1f8f0e41e7702136ac in term 1.
I20260926 11:40:52.778287 14458 raft_consensus.cc:2454] T d6cff98a906f41479dbee0d3ab8073dd P 039d3bd43cdd4aa382e95b24e2401a1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee1c21f2176e4c1f8f0e41e7702136ac in term 1.
I20260926 11:40:52.778518 14109 leader_election.cc:302] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039d3bd43cdd4aa382e95b24e2401a1e, ee1c21f2176e4c1f8f0e41e7702136ac; no voters:
I20260926 11:40:52.778659 14540 raft_consensus.cc:2781] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:52.778868 14540 raft_consensus.cc:686] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac [term 1 LEADER]: Becoming Leader. State: Replica: ee1c21f2176e4c1f8f0e41e7702136ac, State: Running, Role: LEADER
I20260926 11:40:52.779095 14540 consensus_queue.cc:237] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.781255 14017 catalog_manager.cc:5495] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac reported cstate change: term changed from 0 to 1, leader changed from <none> to ee1c21f2176e4c1f8f0e41e7702136ac (127.6.96.1). New cstate: current_term: 1 leader_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } health_report { overall_health: UNKNOWN } } }
I20260926 11:40:52.797887 14603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:52.794361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57004 (local address 127.6.96.1:42589)
0926 11:40:52.794584 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:52.794589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:52.794602 (+ 13us) server_negotiation.cc:407] Connection header received
0926 11:40:52.794676 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:52.794678 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:52.794730 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:52.794853 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:40:52.795108 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.795615 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.796438 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:52.796660 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:52.796744 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:52.796998 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:40:52.797525 (+ 527us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:40:52.797630 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:52.797686 (+ 56us) server_negotiation.cc:299] Negotiation successful
0926 11:40:52.797738 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":58,"threads_started":1}
I20260926 11:40:52.800380 14458 raft_consensus.cc:1261] T d6cff98a906f41479dbee0d3ab8073dd P 039d3bd43cdd4aa382e95b24e2401a1e [term 1 FOLLOWER]: Refusing update from remote peer ee1c21f2176e4c1f8f0e41e7702136ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:52.800467 14318 raft_consensus.cc:1261] T d6cff98a906f41479dbee0d3ab8073dd P 1d08dd932879485aabf14c8b3909a7b9 [term 1 FOLLOWER]: Refusing update from remote peer ee1c21f2176e4c1f8f0e41e7702136ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:52.800813 14540 consensus_queue.cc:1035] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:52.800987 14600 consensus_queue.cc:1035] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:52.812100 14019 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:52.812261 14019 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:52.814209 14019 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375: Sending DeleteTablet for 3 replicas of tablet d6cff98a906f41479dbee0d3ab8073dd
I20260926 11:40:52.814728 14158 tablet_service.cc:1514] Processing DeleteTablet for tablet d6cff98a906f41479dbee0d3ab8073dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:52 UTC) from {username='oryx', principal='oryx/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:56996
I20260926 11:40:52.814913 14298 tablet_service.cc:1514] Processing DeleteTablet for tablet d6cff98a906f41479dbee0d3ab8073dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:52 UTC) from {username='oryx', principal='oryx/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:55486
I20260926 11:40:52.815107 14557 tablet_replica.cc:330] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac: stopping tablet replica
I20260926 11:40:52.815104 14555 tablet_replica.cc:330] T d6cff98a906f41479dbee0d3ab8073dd P 1d08dd932879485aabf14c8b3909a7b9: stopping tablet replica
I20260926 11:40:52.815220 14557 raft_consensus.cc:2229] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac [term 1 LEADER]: Raft consensus shutting down.
I20260926 11:40:52.815243 14555 raft_consensus.cc:2229] T d6cff98a906f41479dbee0d3ab8073dd P 1d08dd932879485aabf14c8b3909a7b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:52.815369 14555 raft_consensus.cc:2258] T d6cff98a906f41479dbee0d3ab8073dd P 1d08dd932879485aabf14c8b3909a7b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:52.815507 14557 raft_consensus.cc:2258] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:52.815694 14438 tablet_service.cc:1514] Processing DeleteTablet for tablet d6cff98a906f41479dbee0d3ab8073dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:52 UTC) from {username='oryx', principal='oryx/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:38152
I20260926 11:40:52.815873 14556 tablet_replica.cc:330] T d6cff98a906f41479dbee0d3ab8073dd P 039d3bd43cdd4aa382e95b24e2401a1e: stopping tablet replica
I20260926 11:40:52.816008 14556 raft_consensus.cc:2229] T d6cff98a906f41479dbee0d3ab8073dd P 039d3bd43cdd4aa382e95b24e2401a1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:52.816134 14556 raft_consensus.cc:2258] T d6cff98a906f41479dbee0d3ab8073dd P 039d3bd43cdd4aa382e95b24e2401a1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:52.816104 14019 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 11:40:52.816311 14555 ts_tablet_manager.cc:1905] T d6cff98a906f41479dbee0d3ab8073dd P 1d08dd932879485aabf14c8b3909a7b9: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 11:40:52.816502 14019 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:40:52.817709 14556 ts_tablet_manager.cc:1905] T d6cff98a906f41479dbee0d3ab8073dd P 039d3bd43cdd4aa382e95b24e2401a1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:52.818924 14555 ts_tablet_manager.cc:1918] T d6cff98a906f41479dbee0d3ab8073dd P 1d08dd932879485aabf14c8b3909a7b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:40:52.818991 14555 log.cc:1196] T d6cff98a906f41479dbee0d3ab8073dd P 1d08dd932879485aabf14c8b3909a7b9: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-1/wal/wals/d6cff98a906f41479dbee0d3ab8073dd
I20260926 11:40:52.819226 14555 ts_tablet_manager.cc:1939] T d6cff98a906f41479dbee0d3ab8073dd P 1d08dd932879485aabf14c8b3909a7b9: Deleting consensus metadata
I20260926 11:40:52.819355 14557 ts_tablet_manager.cc:1905] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:52.820072 14008 catalog_manager.cc:4841] TS 1d08dd932879485aabf14c8b3909a7b9 (127.6.96.2:42263): tablet d6cff98a906f41479dbee0d3ab8073dd (table test-table [id=7f30ffd43d8d4c6cbc50567a19c09a80]) successfully deleted
I20260926 11:40:52.820135 14556 ts_tablet_manager.cc:1918] T d6cff98a906f41479dbee0d3ab8073dd P 039d3bd43cdd4aa382e95b24e2401a1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:40:52.820200 14556 log.cc:1196] T d6cff98a906f41479dbee0d3ab8073dd P 039d3bd43cdd4aa382e95b24e2401a1e: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/d6cff98a906f41479dbee0d3ab8073dd
I20260926 11:40:52.820417 14556 ts_tablet_manager.cc:1939] T d6cff98a906f41479dbee0d3ab8073dd P 039d3bd43cdd4aa382e95b24e2401a1e: Deleting consensus metadata
I20260926 11:40:52.821192 14005 catalog_manager.cc:4841] TS 039d3bd43cdd4aa382e95b24e2401a1e (127.6.96.3:37865): tablet d6cff98a906f41479dbee0d3ab8073dd (table test-table [id=7f30ffd43d8d4c6cbc50567a19c09a80]) successfully deleted
I20260926 11:40:52.821471 14557 ts_tablet_manager.cc:1918] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:40:52.821537 14557 log.cc:1196] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-0/wal/wals/d6cff98a906f41479dbee0d3ab8073dd
I20260926 11:40:52.821743 14557 ts_tablet_manager.cc:1939] T d6cff98a906f41479dbee0d3ab8073dd P ee1c21f2176e4c1f8f0e41e7702136ac: Deleting consensus metadata
I20260926 11:40:52.822435 14006 catalog_manager.cc:4841] TS ee1c21f2176e4c1f8f0e41e7702136ac (127.6.96.1:42589): tablet d6cff98a906f41479dbee0d3ab8073dd (table test-table [id=7f30ffd43d8d4c6cbc50567a19c09a80]) successfully deleted
I20260926 11:40:52.824679 14158 tablet_service.cc:1467] Processing CreateTablet for tablet fac34375deae43b7b415c69446b4da72 (DEFAULT_TABLE table=test-table [id=4c289b7c53a5427daf1905ec11650fdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:52.824815 14438 tablet_service.cc:1467] Processing CreateTablet for tablet fac34375deae43b7b415c69446b4da72 (DEFAULT_TABLE table=test-table [id=4c289b7c53a5427daf1905ec11650fdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:52.824942 14158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fac34375deae43b7b415c69446b4da72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:52.825021 14438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fac34375deae43b7b415c69446b4da72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:52.827186 14535 tablet_bootstrap.cc:492] T fac34375deae43b7b415c69446b4da72 P ee1c21f2176e4c1f8f0e41e7702136ac: Bootstrap starting.
I20260926 11:40:52.827186 14532 tablet_bootstrap.cc:492] T fac34375deae43b7b415c69446b4da72 P 039d3bd43cdd4aa382e95b24e2401a1e: Bootstrap starting.
I20260926 11:40:52.827641 14298 tablet_service.cc:1467] Processing CreateTablet for tablet fac34375deae43b7b415c69446b4da72 (DEFAULT_TABLE table=test-table [id=4c289b7c53a5427daf1905ec11650fdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:40:52.827889 14298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fac34375deae43b7b415c69446b4da72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:52.828064 14532 tablet_bootstrap.cc:654] T fac34375deae43b7b415c69446b4da72 P 039d3bd43cdd4aa382e95b24e2401a1e: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:52.828064 14535 tablet_bootstrap.cc:654] T fac34375deae43b7b415c69446b4da72 P ee1c21f2176e4c1f8f0e41e7702136ac: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:52.829417 14535 tablet_bootstrap.cc:492] T fac34375deae43b7b415c69446b4da72 P ee1c21f2176e4c1f8f0e41e7702136ac: No bootstrap required, opened a new log
I20260926 11:40:52.829501 14535 ts_tablet_manager.cc:1397] T fac34375deae43b7b415c69446b4da72 P ee1c21f2176e4c1f8f0e41e7702136ac: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 11:40:52.829879 14535 raft_consensus.cc:348] T fac34375deae43b7b415c69446b4da72 P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.829973 14535 raft_consensus.cc:374] T fac34375deae43b7b415c69446b4da72 P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:52.830001 14535 raft_consensus.cc:729] T fac34375deae43b7b415c69446b4da72 P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee1c21f2176e4c1f8f0e41e7702136ac, State: Initialized, Role: FOLLOWER
I20260926 11:40:52.830118 14535 consensus_queue.cc:260] T fac34375deae43b7b415c69446b4da72 P ee1c21f2176e4c1f8f0e41e7702136ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.830361 14535 ts_tablet_manager.cc:1428] T fac34375deae43b7b415c69446b4da72 P ee1c21f2176e4c1f8f0e41e7702136ac: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:52.830384 14533 tablet_bootstrap.cc:492] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9: Bootstrap starting.
I20260926 11:40:52.831223 14532 tablet_bootstrap.cc:492] T fac34375deae43b7b415c69446b4da72 P 039d3bd43cdd4aa382e95b24e2401a1e: No bootstrap required, opened a new log
I20260926 11:40:52.831272 14533 tablet_bootstrap.cc:654] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:52.831310 14532 ts_tablet_manager.cc:1397] T fac34375deae43b7b415c69446b4da72 P 039d3bd43cdd4aa382e95b24e2401a1e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260926 11:40:52.831763 14532 raft_consensus.cc:348] T fac34375deae43b7b415c69446b4da72 P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.831867 14532 raft_consensus.cc:374] T fac34375deae43b7b415c69446b4da72 P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:52.831898 14532 raft_consensus.cc:729] T fac34375deae43b7b415c69446b4da72 P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039d3bd43cdd4aa382e95b24e2401a1e, State: Initialized, Role: FOLLOWER
I20260926 11:40:52.832015 14532 consensus_queue.cc:260] T fac34375deae43b7b415c69446b4da72 P 039d3bd43cdd4aa382e95b24e2401a1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.832273 14532 ts_tablet_manager.cc:1428] T fac34375deae43b7b415c69446b4da72 P 039d3bd43cdd4aa382e95b24e2401a1e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:40:52.833220 14533 tablet_bootstrap.cc:492] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9: No bootstrap required, opened a new log
I20260926 11:40:52.833314 14533 ts_tablet_manager.cc:1397] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 11:40:52.833716 14533 raft_consensus.cc:348] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.833828 14533 raft_consensus.cc:374] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:52.833868 14533 raft_consensus.cc:729] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d08dd932879485aabf14c8b3909a7b9, State: Initialized, Role: FOLLOWER
I20260926 11:40:52.833987 14533 consensus_queue.cc:260] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.834235 14533 ts_tablet_manager.cc:1428] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 11:40:52.983759 14550 raft_consensus.cc:482] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:40:52.983897 14550 raft_consensus.cc:504] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.984269 14550 leader_election.cc:288] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee1c21f2176e4c1f8f0e41e7702136ac (127.6.96.1:42589), 039d3bd43cdd4aa382e95b24e2401a1e (127.6.96.3:37865)
I20260926 11:40:52.984687 14178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fac34375deae43b7b415c69446b4da72" candidate_uuid: "1d08dd932879485aabf14c8b3909a7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" is_pre_election: true
I20260926 11:40:52.984696 14458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fac34375deae43b7b415c69446b4da72" candidate_uuid: "1d08dd932879485aabf14c8b3909a7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" is_pre_election: true
I20260926 11:40:52.984825 14178 raft_consensus.cc:2454] T fac34375deae43b7b415c69446b4da72 P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d08dd932879485aabf14c8b3909a7b9 in term 0.
I20260926 11:40:52.984831 14458 raft_consensus.cc:2454] T fac34375deae43b7b415c69446b4da72 P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d08dd932879485aabf14c8b3909a7b9 in term 0.
I20260926 11:40:52.985114 14249 leader_election.cc:302] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 039d3bd43cdd4aa382e95b24e2401a1e, 1d08dd932879485aabf14c8b3909a7b9, ee1c21f2176e4c1f8f0e41e7702136ac; no voters:
I20260926 11:40:52.985292 14550 raft_consensus.cc:2781] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:40:52.985353 14550 raft_consensus.cc:482] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:40:52.985380 14550 raft_consensus.cc:3037] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:52.986266 14550 raft_consensus.cc:504] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.986575 14550 leader_election.cc:288] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9 [CANDIDATE]: Term 1 election: Requested vote from peers ee1c21f2176e4c1f8f0e41e7702136ac (127.6.96.1:42589), 039d3bd43cdd4aa382e95b24e2401a1e (127.6.96.3:37865)
I20260926 11:40:52.986917 14178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fac34375deae43b7b415c69446b4da72" candidate_uuid: "1d08dd932879485aabf14c8b3909a7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac"
I20260926 11:40:52.986961 14458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fac34375deae43b7b415c69446b4da72" candidate_uuid: "1d08dd932879485aabf14c8b3909a7b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039d3bd43cdd4aa382e95b24e2401a1e"
I20260926 11:40:52.987020 14178 raft_consensus.cc:3037] T fac34375deae43b7b415c69446b4da72 P ee1c21f2176e4c1f8f0e41e7702136ac [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:52.987082 14458 raft_consensus.cc:3037] T fac34375deae43b7b415c69446b4da72 P 039d3bd43cdd4aa382e95b24e2401a1e [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:52.987880 14178 raft_consensus.cc:2454] T fac34375deae43b7b415c69446b4da72 P ee1c21f2176e4c1f8f0e41e7702136ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d08dd932879485aabf14c8b3909a7b9 in term 1.
I20260926 11:40:52.987937 14458 raft_consensus.cc:2454] T fac34375deae43b7b415c69446b4da72 P 039d3bd43cdd4aa382e95b24e2401a1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d08dd932879485aabf14c8b3909a7b9 in term 1.
I20260926 11:40:52.988128 14250 leader_election.cc:302] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d08dd932879485aabf14c8b3909a7b9, ee1c21f2176e4c1f8f0e41e7702136ac; no voters:
I20260926 11:40:52.988298 14550 raft_consensus.cc:2781] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:52.988384 14550 raft_consensus.cc:686] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9 [term 1 LEADER]: Becoming Leader. State: Replica: 1d08dd932879485aabf14c8b3909a7b9, State: Running, Role: LEADER
I20260926 11:40:52.988499 14550 consensus_queue.cc:237] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } }
I20260926 11:40:52.990065 14019 catalog_manager.cc:5495] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d08dd932879485aabf14c8b3909a7b9 (127.6.96.2). New cstate: current_term: 1 leader_uuid: "1d08dd932879485aabf14c8b3909a7b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d08dd932879485aabf14c8b3909a7b9" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 } health_report { overall_health: UNKNOWN } } }
I20260926 11:40:53.029166 14528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:53.025859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55508 (local address 127.6.96.2:42263)
0926 11:40:53.025974 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:53.025978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:53.025989 (+ 11us) server_negotiation.cc:407] Connection header received
0926 11:40:53.026166 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:53.026169 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:53.026211 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:53.026268 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:40:53.026579 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:53.027114 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:53.028004 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:53.028220 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:53.028302 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:53.028688 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:40:53.028766 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:40:53.028832 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:53.028975 (+ 143us) server_negotiation.cc:299] Negotiation successful
0926 11:40:53.029021 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 11:40:53.034926 14457 raft_consensus.cc:1261] T fac34375deae43b7b415c69446b4da72 P 039d3bd43cdd4aa382e95b24e2401a1e [term 1 FOLLOWER]: Refusing update from remote peer 1d08dd932879485aabf14c8b3909a7b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:53.034912 14178 raft_consensus.cc:1261] T fac34375deae43b7b415c69446b4da72 P ee1c21f2176e4c1f8f0e41e7702136ac [term 1 FOLLOWER]: Refusing update from remote peer 1d08dd932879485aabf14c8b3909a7b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:40:53.035319 14550 consensus_queue.cc:1035] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee1c21f2176e4c1f8f0e41e7702136ac" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:53.035490 14542 consensus_queue.cc:1035] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039d3bd43cdd4aa382e95b24e2401a1e" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:40:53.064896 14017 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:53.065045 14017 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:40:53.066733 14017 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9cb6f434259e4f0d90652f2e1895b375: Sending DeleteTablet for 3 replicas of tablet fac34375deae43b7b415c69446b4da72
I20260926 11:40:53.067221 14158 tablet_service.cc:1514] Processing DeleteTablet for tablet fac34375deae43b7b415c69446b4da72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:53 UTC) from {username='oryx', principal='oryx/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:56996
I20260926 11:40:53.067272 14298 tablet_service.cc:1514] Processing DeleteTablet for tablet fac34375deae43b7b415c69446b4da72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:53 UTC) from {username='oryx', principal='oryx/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:55486
I20260926 11:40:53.067396 14557 tablet_replica.cc:330] T fac34375deae43b7b415c69446b4da72 P ee1c21f2176e4c1f8f0e41e7702136ac: stopping tablet replica
I20260926 11:40:53.067380 14438 tablet_service.cc:1514] Processing DeleteTablet for tablet fac34375deae43b7b415c69446b4da72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:40:53 UTC) from {username='oryx', principal='oryx/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:38152
I20260926 11:40:53.067437 14555 tablet_replica.cc:330] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9: stopping tablet replica
I20260926 11:40:53.067521 14556 tablet_replica.cc:330] T fac34375deae43b7b415c69446b4da72 P 039d3bd43cdd4aa382e95b24e2401a1e: stopping tablet replica
I20260926 11:40:53.067545 14555 raft_consensus.cc:2229] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 11:40:53.067554 14557 raft_consensus.cc:2229] T fac34375deae43b7b415c69446b4da72 P ee1c21f2176e4c1f8f0e41e7702136ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:53.067644 14556 raft_consensus.cc:2229] T fac34375deae43b7b415c69446b4da72 P 039d3bd43cdd4aa382e95b24e2401a1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:40:53.067709 14557 raft_consensus.cc:2258] T fac34375deae43b7b415c69446b4da72 P ee1c21f2176e4c1f8f0e41e7702136ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:53.067739 14555 raft_consensus.cc:2258] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:53.067749 14556 raft_consensus.cc:2258] T fac34375deae43b7b415c69446b4da72 P 039d3bd43cdd4aa382e95b24e2401a1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:40:53.068362 14557 ts_tablet_manager.cc:1905] T fac34375deae43b7b415c69446b4da72 P ee1c21f2176e4c1f8f0e41e7702136ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:53.068373 14555 ts_tablet_manager.cc:1905] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:53.068394 14556 ts_tablet_manager.cc:1905] T fac34375deae43b7b415c69446b4da72 P 039d3bd43cdd4aa382e95b24e2401a1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:40:53.070463 14557 ts_tablet_manager.cc:1918] T fac34375deae43b7b415c69446b4da72 P ee1c21f2176e4c1f8f0e41e7702136ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 11:40:53.070531 14557 log.cc:1196] T fac34375deae43b7b415c69446b4da72 P ee1c21f2176e4c1f8f0e41e7702136ac: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-0/wal/wals/fac34375deae43b7b415c69446b4da72
I20260926 11:40:53.070716 14556 ts_tablet_manager.cc:1918] T fac34375deae43b7b415c69446b4da72 P 039d3bd43cdd4aa382e95b24e2401a1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 11:40:53.070739 14555 ts_tablet_manager.cc:1918] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 11:40:53.070753 14557 ts_tablet_manager.cc:1939] T fac34375deae43b7b415c69446b4da72 P ee1c21f2176e4c1f8f0e41e7702136ac: Deleting consensus metadata
I20260926 11:40:53.070788 14556 log.cc:1196] T fac34375deae43b7b415c69446b4da72 P 039d3bd43cdd4aa382e95b24e2401a1e: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/fac34375deae43b7b415c69446b4da72
I20260926 11:40:53.070793 14555 log.cc:1196] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790422817500949-6528-0/minicluster-data/ts-1/wal/wals/fac34375deae43b7b415c69446b4da72
I20260926 11:40:53.071049 14556 ts_tablet_manager.cc:1939] T fac34375deae43b7b415c69446b4da72 P 039d3bd43cdd4aa382e95b24e2401a1e: Deleting consensus metadata
I20260926 11:40:53.071085 14555 ts_tablet_manager.cc:1939] T fac34375deae43b7b415c69446b4da72 P 1d08dd932879485aabf14c8b3909a7b9: Deleting consensus metadata
I20260926 11:40:53.071384 14006 catalog_manager.cc:4841] TS ee1c21f2176e4c1f8f0e41e7702136ac (127.6.96.1:42589): tablet fac34375deae43b7b415c69446b4da72 (table test-table [id=4c289b7c53a5427daf1905ec11650fdf]) successfully deleted
I20260926 11:40:53.071604 14005 catalog_manager.cc:4841] TS 039d3bd43cdd4aa382e95b24e2401a1e (127.6.96.3:37865): tablet fac34375deae43b7b415c69446b4da72 (table test-table [id=4c289b7c53a5427daf1905ec11650fdf]) successfully deleted
I20260926 11:40:53.071650 14008 catalog_manager.cc:4841] TS 1d08dd932879485aabf14c8b3909a7b9 (127.6.96.2:42263): tablet fac34375deae43b7b415c69446b4da72 (table test-table [id=4c289b7c53a5427daf1905ec11650fdf]) successfully deleted
I20260926 11:40:53.072250 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 14093
I20260926 11:40:53.078462 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 14233
I20260926 11:40:53.084311 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 14373
I20260926 11:40:53.089085 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 13989
2026-09-26T11:40:53Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2905 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:40:53 dist-test-slave-xxxx krb5kdc[14618](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:40:53 dist-test-slave-xxxx krb5kdc[14618](info): set up 2 sockets
Sep 26 11:40:53 dist-test-slave-xxxx krb5kdc[14618](info): commencing operation
krb5kdc: starting...
W20260926 11:40:54.051144 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.941s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:40:54 dist-test-slave-xxxx krb5kdc[14618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:40:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:40:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:40:54.170282 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:39731
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:43885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:39731
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:54.278523 14631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:54.278785 14631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:54.278848 14631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:54.282799 14631 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:40:54.282886 14631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:54.282904 14631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:54.282919 14631 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:40:54.282932 14631 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:40:54.288287 14631 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:43885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:39731
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:39731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:54.289348 14631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:54.290328 14631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:54.298389 14636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:54.298378 14637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:54.298367 14639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:54.298736 14631 server_base.cc:1034] running on GCE node
I20260926 11:40:54.299316 14631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:54.300393 14631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:54.301548 14631 hybrid_clock.cc:648] HybridClock initialized: now 1790422854301533 us; error 25 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-26T11:40:54Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1219 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:40:54 dist-test-slave-xxxx krb5kdc[14644](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:40:54 dist-test-slave-xxxx krb5kdc[14644](info): set up 2 sockets
Sep 26 11:40:54 dist-test-slave-xxxx krb5kdc[14644](info): commencing operation
krb5kdc: starting...
W20260926 11:40:55.263095 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:40:55 dist-test-slave-xxxx krb5kdc[14644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:40:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:40:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:40:55.381626 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:41081
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:34403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:41081
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:55.489296 14657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:55.489573 14657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:55.489629 14657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:55.493981 14657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:40:55.494066 14657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:55.494095 14657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:55.494117 14657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:40:55.494138 14657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:40:55.499675 14657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:34403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:41081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:41081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:55.500722 14657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:55.501771 14657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:55.510003 14665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:55.510048 14657 server_base.cc:1034] running on GCE node
W20260926 11:40:55.510227 14663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:55.510106 14662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:55.510815 14657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:55.512012 14657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:55.513171 14657 hybrid_clock.cc:648] HybridClock initialized: now 1790422855513155 us; error 34 us; skew 500 ppm
Sep 26 11:40:55 dist-test-slave-xxxx krb5kdc[14644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422855, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:55.515792 14657 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:40:55.516610 14657 webserver.cc:460] Webserver started at http://127.6.96.62:42397/ using document root <none> and password file <none>
I20260926 11:40:55.517127 14657 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:55.517180 14657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:55.517351 14657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:55.519348 14657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "7a5c8100151049c8bcba31b9ee1e8414"
format_stamp: "Formatted at 2026-09-26 11:40:55 on dist-test-slave-xxxx"
server_key: "882db4feb1ca36aa0fe92153bd05e679"
server_key_iv: "7881f08be57a62fff02a721134638deb"
server_key_version: "encryptionkey@0"
I20260926 11:40:55.519829 14657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "7a5c8100151049c8bcba31b9ee1e8414"
format_stamp: "Formatted at 2026-09-26 11:40:55 on dist-test-slave-xxxx"
server_key: "882db4feb1ca36aa0fe92153bd05e679"
server_key_iv: "7881f08be57a62fff02a721134638deb"
server_key_version: "encryptionkey@0"
I20260926 11:40:55.523242 14657 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:40:55.525463 14672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:55.526616 14657 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:55.526708 14657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "7a5c8100151049c8bcba31b9ee1e8414"
format_stamp: "Formatted at 2026-09-26 11:40:55 on dist-test-slave-xxxx"
server_key: "882db4feb1ca36aa0fe92153bd05e679"
server_key_iv: "7881f08be57a62fff02a721134638deb"
server_key_version: "encryptionkey@0"
I20260926 11:40:55.526794 14657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:55.538440 14657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:55.541100 14657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:55.541247 14657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:55.547530 14657 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:41081
I20260926 11:40:55.547562 14724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:41081 every 8 connection(s)
I20260926 11:40:55.548496 14657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:40:55.551270 14725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:40:55.556479 14725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414: Bootstrap starting.
I20260926 11:40:55.557647 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 14657
I20260926 11:40:55.557785 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:40:55.558066 6528 external_mini_cluster.cc:1428] Setting key a2079ed49be01c8025c30b79972fcc53
I20260926 11:40:55.559180 14725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414: Neither blocks nor log segments found. Creating new log.
I20260926 11:40:55.559942 14725 log.cc:824] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:55.561699 14725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414: No bootstrap required, opened a new log
I20260926 11:40:55.565279 14725 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a5c8100151049c8bcba31b9ee1e8414" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 41081 } }
I20260926 11:40:55.565497 14725 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:40:55.565550 14725 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a5c8100151049c8bcba31b9ee1e8414, State: Initialized, Role: FOLLOWER
I20260926 11:40:55.565972 14725 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a5c8100151049c8bcba31b9ee1e8414" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 41081 } }
I20260926 11:40:55.566099 14725 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:40:55.566162 14725 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Sep 26 11:40:55 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:55.566264 14725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:40:55.567219 14725 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a5c8100151049c8bcba31b9ee1e8414" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 41081 } }
I20260926 11:40:55.567585 14725 leader_election.cc:302] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a5c8100151049c8bcba31b9ee1e8414; no voters:
I20260926 11:40:55.567919 14725 leader_election.cc:288] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:40:55.567993 14730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:40:55.568190 14730 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [term 1 LEADER]: Becoming Leader. State: Replica: 7a5c8100151049c8bcba31b9ee1e8414, State: Running, Role: LEADER
I20260926 11:40:55.568534 14730 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a5c8100151049c8bcba31b9ee1e8414" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 41081 } }
I20260926 11:40:55.569190 14725 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:40:55.570122 14731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a5c8100151049c8bcba31b9ee1e8414" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a5c8100151049c8bcba31b9ee1e8414" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 41081 } } }
I20260926 11:40:55.570251 14731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:55.570447 14732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a5c8100151049c8bcba31b9ee1e8414. Latest consensus state: current_term: 1 leader_uuid: "7a5c8100151049c8bcba31b9ee1e8414" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a5c8100151049c8bcba31b9ee1e8414" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 41081 } } }
I20260926 11:40:55.570612 14732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:55.572125 14728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:55.559208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49266 (local address 127.6.96.62:41081)
0926 11:40:55.559610 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:55.559626 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:55.559660 (+ 34us) server_negotiation.cc:407] Connection header received
0926 11:40:55.560334 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:55.560370 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:55.560766 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:55.561066 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:55.561810 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:55.562859 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:55.563726 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:55.564201 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:55.564343 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:55.566757 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:55.566762 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:55.566775 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:55.566802 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:55.568574 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:55.569014 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:55.569019 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:55.569024 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:55.569128 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:55.569412 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:55.569415 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:55.569417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:55.569922 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:55.570104 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:55.570964 (+ 860us) server_negotiation.cc:299] Negotiation successful
0926 11:40:55.571263 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":125,"threads_started":1}
W20260926 11:40:55.574249 14746 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 11:40:55.574308 14746 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 11:40:55.574376 14739 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:40:55.575169 14739 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:40:55.580361 14739 catalog_manager.cc:1347] Generated new cluster ID: d69891641bfe4d31b5f9be89491bd4d7
I20260926 11:40:55.580498 14739 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:40:55.601727 14739 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:40:55.603044 14739 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:40:55.608390 14739 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414: Generated new TSK 0
I20260926 11:40:55.609094 14739 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.96.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
I20260926 11:40:55.639722 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:41081
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:34403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:55.748524 14751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:55.748803 14751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:55.748845 14751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:55.753002 14751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:55.753151 14751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:55.753254 14751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:40:55.758941 14751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:34403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:41081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:55.759982 14751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:55.760975 14751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:55.770751 14756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:55.770740 14757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:55.770886 14751 server_base.cc:1034] running on GCE node
W20260926 11:40:55.770743 14759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:55.771404 14751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:55.772058 14751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:55.773204 14751 hybrid_clock.cc:648] HybridClock initialized: now 1790422855773174 us; error 43 us; skew 500 ppm
Sep 26 11:40:55 dist-test-slave-xxxx krb5kdc[14644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422855, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:55.776089 14751 init.cc:377] Logged in from keytab as kudu/127.6.96.1@KRBTEST.COM (short username kudu)
I20260926 11:40:55.776945 14751 webserver.cc:460] Webserver started at http://127.6.96.1:44833/ using document root <none> and password file <none>
I20260926 11:40:55.777485 14751 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:55.777549 14751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:55.777729 14751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:55.779767 14751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "dce8698f4ef745f8bb4a7e6d4762dbb1"
format_stamp: "Formatted at 2026-09-26 11:40:55 on dist-test-slave-xxxx"
server_key: "758709b68ea1a7784373cd256053bfc7"
server_key_iv: "93fc2abc9ef4d144f7b3bbdcd7d8aa96"
server_key_version: "encryptionkey@0"
I20260926 11:40:55.780344 14751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "dce8698f4ef745f8bb4a7e6d4762dbb1"
format_stamp: "Formatted at 2026-09-26 11:40:55 on dist-test-slave-xxxx"
server_key: "758709b68ea1a7784373cd256053bfc7"
server_key_iv: "93fc2abc9ef4d144f7b3bbdcd7d8aa96"
server_key_version: "encryptionkey@0"
I20260926 11:40:55.784089 14751 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 11:40:55.786684 14766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:55.788009 14751 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:40:55.788125 14751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "dce8698f4ef745f8bb4a7e6d4762dbb1"
format_stamp: "Formatted at 2026-09-26 11:40:55 on dist-test-slave-xxxx"
server_key: "758709b68ea1a7784373cd256053bfc7"
server_key_iv: "93fc2abc9ef4d144f7b3bbdcd7d8aa96"
server_key_version: "encryptionkey@0"
I20260926 11:40:55.788247 14751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:55.805565 14751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:55.808831 14751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:55.809015 14751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:55.809633 14751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:55.810475 14751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:55.810525 14751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:55.810592 14751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:55.810622 14751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:55.819303 14751 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:39443
I20260926 11:40:55.819356 14879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:39443 every 8 connection(s)
I20260926 11:40:55.820217 14751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:40:55.825888 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 14751
I20260926 11:40:55.826026 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:40:55.826296 6528 external_mini_cluster.cc:1428] Setting key 5fad239ca48b8d526959e70f4a7995ed
Sep 26 11:40:55 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422855, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:55.834344 14728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:55.822036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:37861 (local address 127.6.96.62:41081)
0926 11:40:55.822165 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:55.822169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:55.823321 (+ 1152us) server_negotiation.cc:407] Connection header received
0926 11:40:55.824210 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:55.824213 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:55.824262 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:55.824338 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:55.825502 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:55.826299 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:55.827542 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:55.827777 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:55.827865 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:55.830465 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:55.830469 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:55.830472 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:55.830500 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:55.832236 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:55.832755 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:55.832760 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:55.832761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:55.832859 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:55.833177 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:55.833181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:55.833183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:55.833415 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:55.833522 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:55.834042 (+ 520us) server_negotiation.cc:299] Negotiation successful
0926 11:40:55.834174 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 11:40:55.835196 14882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:55.822446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.1:37861)
0926 11:40:55.823161 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:55.823220 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:55.824007 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:55.824482 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:55.824489 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:55.824840 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:55.825347 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:55.825361 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:55.826448 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:55.826451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:55.827424 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:55.827435 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:55.827916 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:55.827920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:55.828083 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:55.828616 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:55.828633 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:55.830286 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:55.832376 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:55.832381 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:55.832395 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:55.832660 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:55.832952 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:55.832954 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:55.832956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:55.833095 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:55.833508 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:55.833513 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:55.833794 (+ 281us) client_negotiation.cc:769] Sending connection context
0926 11:40:55.833960 (+ 166us) client_negotiation.cc:240] Negotiation successful
0926 11:40:55.834231 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":123,"threads_started":1}
WARNING: no policy specified for kudu/127.6.96.2@KRBTEST.COM; defaulting to no policy
I20260926 11:40:55.836381 14880 heartbeater.cc:347] Connected to a master server at 127.6.96.62:41081
I20260926 11:40:55.836618 14880 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:55.837118 14880 heartbeater.cc:511] Master 127.6.96.62:41081 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.2@KRBTEST.COM" created.
I20260926 11:40:55.838676 14689 ts_manager.cc:194] Registered new tserver with Master: dce8698f4ef745f8bb4a7e6d4762dbb1 (127.6.96.1:39443)
I20260926 11:40:55.840073 14689 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.1@KRBTEST.COM'} at 127.6.96.1:37861
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
I20260926 11:40:55.854744 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:41081
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:34403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:55.963744 14885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:55.964017 14885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:55.964061 14885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:55.968288 14885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:55.968356 14885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:55.968417 14885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:40:55.973925 14885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:34403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:41081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:55.975057 14885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:55.976043 14885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:55.985338 14891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:55.985342 14893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:55.985342 14890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:55.985666 14885 server_base.cc:1034] running on GCE node
I20260926 11:40:55.986085 14885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:55.986738 14885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:55.987896 14885 hybrid_clock.cc:648] HybridClock initialized: now 1790422855987876 us; error 30 us; skew 500 ppm
Sep 26 11:40:55 dist-test-slave-xxxx krb5kdc[14644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422855, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:55.990758 14885 init.cc:377] Logged in from keytab as kudu/127.6.96.2@KRBTEST.COM (short username kudu)
I20260926 11:40:55.991721 14885 webserver.cc:460] Webserver started at http://127.6.96.2:36845/ using document root <none> and password file <none>
I20260926 11:40:55.992256 14885 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:55.992324 14885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:55.992517 14885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:55.994489 14885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "ca9f684414ba4c20a09436dab48e8bc3"
format_stamp: "Formatted at 2026-09-26 11:40:55 on dist-test-slave-xxxx"
server_key: "5725f37368283aada2282cf247d899e0"
server_key_iv: "c527c036cd5a358300d2e8392ca7a288"
server_key_version: "encryptionkey@0"
I20260926 11:40:55.995042 14885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "ca9f684414ba4c20a09436dab48e8bc3"
format_stamp: "Formatted at 2026-09-26 11:40:55 on dist-test-slave-xxxx"
server_key: "5725f37368283aada2282cf247d899e0"
server_key_iv: "c527c036cd5a358300d2e8392ca7a288"
server_key_version: "encryptionkey@0"
I20260926 11:40:55.998548 14885 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:40:56.000943 14900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:56.002146 14885 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:40:56.002243 14885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "ca9f684414ba4c20a09436dab48e8bc3"
format_stamp: "Formatted at 2026-09-26 11:40:55 on dist-test-slave-xxxx"
server_key: "5725f37368283aada2282cf247d899e0"
server_key_iv: "c527c036cd5a358300d2e8392ca7a288"
server_key_version: "encryptionkey@0"
I20260926 11:40:56.002347 14885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:56.015556 14885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:56.018340 14885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:56.018507 14885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:56.019136 14885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:56.019945 14885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:56.019992 14885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:56.020035 14885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:56.020102 14885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:56.028558 14885 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:34741
I20260926 11:40:56.028589 15013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:34741 every 8 connection(s)
I20260926 11:40:56.029451 14885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:40:56.030385 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 14885
I20260926 11:40:56.030489 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:40:56.030771 6528 external_mini_cluster.cc:1428] Setting key 7d0fd95942021087880206d86df2b3ca
Sep 26 11:40:56 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422855, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.96.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.3@KRBTEST.COM" created.
I20260926 11:40:56.044238 14728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:56.031319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:54691 (local address 127.6.96.62:41081)
0926 11:40:56.031458 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:56.031463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:56.032705 (+ 1242us) server_negotiation.cc:407] Connection header received
0926 11:40:56.033557 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:56.033561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:56.033617 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:56.033704 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:56.034904 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.035656 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.036881 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.037235 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.037357 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:56.040061 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:56.040066 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:56.040069 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:56.040103 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:56.041659 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:56.042377 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:56.042381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:56.042383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:56.042468 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:56.042880 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:56.042885 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:56.042887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:56.043146 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:56.043270 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:56.043918 (+ 648us) server_negotiation.cc:299] Negotiation successful
0926 11:40:56.044065 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 11:40:56.045233 15016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:56.031792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.2:54691)
0926 11:40:56.032567 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:56.032607 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:56.033361 (+ 754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:56.033894 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:56.033901 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:56.034248 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:56.034743 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:56.034756 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.035784 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.035788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:56.036727 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:56.036737 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.037414 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.037417 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:56.037573 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:56.038147 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:56.038164 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:56.039824 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:56.041827 (+ 2003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:56.041834 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:56.041871 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:56.042199 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:56.042557 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:56.042560 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:56.042561 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:56.042738 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:56.043291 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:56.043298 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:56.043646 (+ 348us) client_negotiation.cc:769] Sending connection context
0926 11:40:56.043830 (+ 184us) client_negotiation.cc:240] Negotiation successful
0926 11:40:56.044126 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":107,"threads_started":1}
I20260926 11:40:56.046423 15014 heartbeater.cc:347] Connected to a master server at 127.6.96.62:41081
I20260926 11:40:56.046666 15014 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:56.047216 15014 heartbeater.cc:511] Master 127.6.96.62:41081 requested a full tablet report, sending...
I20260926 11:40:56.048341 14689 ts_manager.cc:194] Registered new tserver with Master: ca9f684414ba4c20a09436dab48e8bc3 (127.6.96.2:34741)
I20260926 11:40:56.049134 14689 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.2@KRBTEST.COM'} at 127.6.96.2:54691
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
I20260926 11:40:56.058420 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:41081
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:34403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:56.166407 15019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:56.166685 15019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:56.166728 15019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:56.170889 15019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:56.170954 15019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:56.171013 15019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:40:56.176410 15019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:34403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:41081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:56.177454 15019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:56.178426 15019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:56.187772 15027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:56.187799 15024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:56.187804 15025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:56.187965 15019 server_base.cc:1034] running on GCE node
I20260926 11:40:56.188494 15019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:56.189112 15019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:56.190286 15019 hybrid_clock.cc:648] HybridClock initialized: now 1790422856190254 us; error 51 us; skew 500 ppm
Sep 26 11:40:56 dist-test-slave-xxxx krb5kdc[14644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422856, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:56.193168 15019 init.cc:377] Logged in from keytab as kudu/127.6.96.3@KRBTEST.COM (short username kudu)
I20260926 11:40:56.194104 15019 webserver.cc:460] Webserver started at http://127.6.96.3:38305/ using document root <none> and password file <none>
I20260926 11:40:56.194746 15019 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:56.194849 15019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:56.195065 15019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:40:56.197086 15019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "e398b5ee88f44f66995f4215fe0f293f"
format_stamp: "Formatted at 2026-09-26 11:40:56 on dist-test-slave-xxxx"
server_key: "4cbe908e7dbf539c0eb5936078afddaf"
server_key_iv: "a479ed045578ef19fb4e90c557f31ba4"
server_key_version: "encryptionkey@0"
I20260926 11:40:56.197620 15019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "e398b5ee88f44f66995f4215fe0f293f"
format_stamp: "Formatted at 2026-09-26 11:40:56 on dist-test-slave-xxxx"
server_key: "4cbe908e7dbf539c0eb5936078afddaf"
server_key_iv: "a479ed045578ef19fb4e90c557f31ba4"
server_key_version: "encryptionkey@0"
I20260926 11:40:56.201222 15019 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260926 11:40:56.203528 15034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:56.204731 15019 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 11:40:56.204861 15019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "e398b5ee88f44f66995f4215fe0f293f"
format_stamp: "Formatted at 2026-09-26 11:40:56 on dist-test-slave-xxxx"
server_key: "4cbe908e7dbf539c0eb5936078afddaf"
server_key_iv: "a479ed045578ef19fb4e90c557f31ba4"
server_key_version: "encryptionkey@0"
I20260926 11:40:56.204953 15019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:56.220602 15019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:56.223778 15019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:56.223995 15019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:56.224609 15019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:56.225397 15019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:56.225441 15019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:56.225479 15019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:56.225499 15019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:56.233989 15019 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:45911
I20260926 11:40:56.234018 15147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:45911 every 8 connection(s)
I20260926 11:40:56.235021 15019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:40:56.244508 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 15019
I20260926 11:40:56.244609 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
Sep 26 11:40:56 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422856, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:40:56.244885 6528 external_mini_cluster.cc:1428] Setting key 6694baa4579579b6249fb94a5285f785
I20260926 11:40:56.248752 14728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:56.236930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:53803 (local address 127.6.96.62:41081)
0926 11:40:56.237057 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:56.237060 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:56.238031 (+ 971us) server_negotiation.cc:407] Connection header received
0926 11:40:56.239298 (+ 1267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:56.239302 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:56.239360 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:56.239435 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:56.240714 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.241232 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.242380 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.242592 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.242679 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:56.245297 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:56.245301 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:56.245303 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:56.245329 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:56.246807 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:56.247413 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:56.247416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:56.247417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:56.247494 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:56.247865 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:56.247867 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:56.247868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:56.248046 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:56.248117 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:56.248547 (+ 430us) server_negotiation.cc:299] Negotiation successful
0926 11:40:56.248637 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 11:40:56.249596 15150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:56.237360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.3:53803)
0926 11:40:56.237869 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:56.237913 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:56.239077 (+ 1164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:56.239622 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:56.239630 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:56.240045 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:56.240557 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:56.240571 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.241345 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.241348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:56.242257 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:56.242266 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.242732 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.242736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:56.242977 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:56.243549 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:56.243572 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:56.245107 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:40:56.246957 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:56.246964 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:56.246981 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:56.247321 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:56.247591 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:40:56.247593 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:40:56.247595 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:40:56.247772 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:40:56.248118 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:40:56.248123 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:40:56.248363 (+ 240us) client_negotiation.cc:769] Sending connection context
0926 11:40:56.248517 (+ 154us) client_negotiation.cc:240] Negotiation successful
0926 11:40:56.248773 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":146,"threads_started":1}
I20260926 11:40:56.250674 15148 heartbeater.cc:347] Connected to a master server at 127.6.96.62:41081
I20260926 11:40:56.250917 15148 heartbeater.cc:464] Registering TS with master...
I20260926 11:40:56.251380 15148 heartbeater.cc:511] Master 127.6.96.62:41081 requested a full tablet report, sending...
I20260926 11:40:56.252334 14689 ts_manager.cc:194] Registered new tserver with Master: e398b5ee88f44f66995f4215fe0f293f (127.6.96.3:45911)
I20260926 11:40:56.252851 14689 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.3@KRBTEST.COM'} at 127.6.96.3:53803
I20260926 11:40:56.258809 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.96.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.96.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.96.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.1.keytab.
Entry for principal oryx/127.6.96.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.1.keytab.
WARNING: no policy specified for oryx/127.6.96.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.96.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.96.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.2.keytab.
Entry for principal oryx/127.6.96.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.2.keytab.
WARNING: no policy specified for oryx/127.6.96.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.96.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.96.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.3.keytab.
Entry for principal oryx/127.6.96.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.3.keytab.
I20260926 11:40:56.328831 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 14751
I20260926 11:40:56.333516 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 14885
I20260926 11:40:56.337648 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 15019
I20260926 11:40:56.341727 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 14657
I20260926 11:40:56.346939 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:41081
--webserver_interface=127.6.96.62
--webserver_port=42397
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:34403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:41081
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:40:56.451834 15157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:56.452103 15157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:56.452159 15157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:56.456113 15157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:40:56.456166 15157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:56.456190 15157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:56.456211 15157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:40:56.456239 15157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:40:56.461678 15157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:34403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:41081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:41081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=42397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:56.462695 15157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:56.463706 15157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:56.471653 15165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:56.471925 15162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:56.472019 15163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:56.472064 15157 server_base.cc:1034] running on GCE node
I20260926 11:40:56.472581 15157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:56.473639 15157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:56.474787 15157 hybrid_clock.cc:648] HybridClock initialized: now 1790422856474779 us; error 40 us; skew 500 ppm
Sep 26 11:40:56 dist-test-slave-xxxx krb5kdc[14644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422856, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:56.482190 15157 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:40:56.483155 15157 webserver.cc:460] Webserver started at http://127.6.96.62:42397/ using document root <none> and password file <none>
I20260926 11:40:56.483682 15157 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:56.483778 15157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:56.487722 15157 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260926 11:40:56.489840 15172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:56.491156 15157 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:40:56.491262 15157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "7a5c8100151049c8bcba31b9ee1e8414"
format_stamp: "Formatted at 2026-09-26 11:40:55 on dist-test-slave-xxxx"
server_key: "882db4feb1ca36aa0fe92153bd05e679"
server_key_iv: "7881f08be57a62fff02a721134638deb"
server_key_version: "encryptionkey@0"
I20260926 11:40:56.491832 15157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:56.512434 15157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:56.515591 15157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:56.515776 15157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:56.522570 15157 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:41081
I20260926 11:40:56.522598 15224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:41081 every 8 connection(s)
I20260926 11:40:56.523641 15157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:40:56.529368 15225 sys_catalog.cc:263] Verifying existing consensus state
I20260926 11:40:56.531805 15225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414: Bootstrap starting.
I20260926 11:40:56.532559 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 15157
I20260926 11:40:56.532982 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:39443
--local_ip_for_outbound_sockets=127.6.96.1
--tserver_master_addrs=127.6.96.62:41081
--webserver_port=44833
--webserver_interface=127.6.96.1
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:34403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.96.1
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 11:40:56.554785 15225 log.cc:824] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414: Log is configured to *not* fsync() on all Append() calls
I20260926 11:40:56.563341 15225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 11:40:56.563733 15225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414: Bootstrap complete.
I20260926 11:40:56.569072 15225 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a5c8100151049c8bcba31b9ee1e8414" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 41081 } }
I20260926 11:40:56.569568 15225 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a5c8100151049c8bcba31b9ee1e8414, State: Initialized, Role: FOLLOWER
I20260926 11:40:56.570145 15225 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a5c8100151049c8bcba31b9ee1e8414" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 41081 } }
I20260926 11:40:56.570293 15225 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:40:56.570358 15225 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:40:56.570441 15225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [term 1 FOLLOWER]: Advancing to term 2
I20260926 11:40:56.572968 15225 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a5c8100151049c8bcba31b9ee1e8414" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 41081 } }
I20260926 11:40:56.573328 15225 leader_election.cc:302] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a5c8100151049c8bcba31b9ee1e8414; no voters:
I20260926 11:40:56.573662 15225 leader_election.cc:288] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 11:40:56.573901 15229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 11:40:56.574182 15229 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [term 2 LEADER]: Becoming Leader. State: Replica: 7a5c8100151049c8bcba31b9ee1e8414, State: Running, Role: LEADER
I20260926 11:40:56.574569 15229 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a5c8100151049c8bcba31b9ee1e8414" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 41081 } }
I20260926 11:40:56.574939 15225 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:40:56.575587 15229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a5c8100151049c8bcba31b9ee1e8414. Latest consensus state: current_term: 2 leader_uuid: "7a5c8100151049c8bcba31b9ee1e8414" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a5c8100151049c8bcba31b9ee1e8414" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 41081 } } }
I20260926 11:40:56.575697 15229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:56.576288 15230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7a5c8100151049c8bcba31b9ee1e8414" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a5c8100151049c8bcba31b9ee1e8414" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 41081 } } }
I20260926 11:40:56.576388 15230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414 [sys.catalog]: This master's current role is: LEADER
I20260926 11:40:56.579586 15245 catalog_manager.cc:1259] Loaded cluster ID: d69891641bfe4d31b5f9be89491bd4d7
I20260926 11:40:56.579633 15245 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414: loading cluster ID for follower catalog manager: success
I20260926 11:40:56.581348 15245 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414: acquiring CA information for follower catalog manager: success
I20260926 11:40:56.582448 15245 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 11:40:56.582564 15236 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:40:56.583294 15236 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:40:56.583650 15236 catalog_manager.cc:1259] Loaded cluster ID: d69891641bfe4d31b5f9be89491bd4d7
I20260926 11:40:56.583694 15236 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:40:56.584183 15236 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:40:56.584542 15236 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7a5c8100151049c8bcba31b9ee1e8414: Loaded TSK: 0
I20260926 11:40:56.585117 15236 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 11:40:56.678315 15227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:56.678587 15227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:56.678659 15227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:56.682883 15227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:56.682960 15227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:56.683043 15227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:40:56.688329 15227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:34403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.1.keytab
--principal=oryx/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:39443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=44833
--tserver_master_addrs=127.6.96.62:41081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:56.689343 15227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:56.690331 15227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:56.699838 15251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:56.699847 15252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:56.699942 15254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:56.700129 15227 server_base.cc:1034] running on GCE node
I20260926 11:40:56.700677 15227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:56.701515 15227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:56.702719 15227 hybrid_clock.cc:648] HybridClock initialized: now 1790422856702694 us; error 34 us; skew 500 ppm
Sep 26 11:40:56 dist-test-slave-xxxx krb5kdc[14644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422856, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:56.706110 15227 init.cc:377] Logged in from keytab as oryx/127.6.96.1@KRBTEST.COM (short username oryx)
I20260926 11:40:56.707015 15227 webserver.cc:460] Webserver started at http://127.6.96.1:44833/ using document root <none> and password file <none>
I20260926 11:40:56.707549 15227 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:56.707618 15227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:56.711604 15227 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 11:40:56.713688 15261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:56.715003 15227 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:40:56.715106 15227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "dce8698f4ef745f8bb4a7e6d4762dbb1"
format_stamp: "Formatted at 2026-09-26 11:40:55 on dist-test-slave-xxxx"
server_key: "758709b68ea1a7784373cd256053bfc7"
server_key_iv: "93fc2abc9ef4d144f7b3bbdcd7d8aa96"
server_key_version: "encryptionkey@0"
I20260926 11:40:56.715613 15227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:56.736862 15227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:56.739902 15227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:56.740078 15227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:56.740643 15227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:56.741438 15227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:56.741483 15227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:56.741534 15227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:56.741559 15227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:56.749967 15227 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:39443
I20260926 11:40:56.750005 15374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:39443 every 8 connection(s)
I20260926 11:40:56.751073 15227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:40:56.760358 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 15227
I20260926 11:40:56.760749 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:34741
--local_ip_for_outbound_sockets=127.6.96.2
--tserver_master_addrs=127.6.96.62:41081
--webserver_port=36845
--webserver_interface=127.6.96.2
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:34403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.96.2
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 11:40:56 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:40:56 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
W20260926 11:40:56.764016 15377 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:56.753157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:43071 (local address 127.6.96.62:41081)
0926 11:40:56.753559 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:56.753575 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:56.753878 (+ 303us) server_negotiation.cc:407] Connection header received
0926 11:40:56.755331 (+ 1453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:56.755342 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:56.755637 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:56.755918 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:56.756933 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.757913 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.759127 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.759614 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.759741 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:56.763330 (+ 3589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.1:43071: BlockingRecv error: recv got EOF from 127.6.96.1:43071 (error 108)
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":99,"threads_started":1}
I20260926 11:40:56.764060 15378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:56.753262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.1:43071)
0926 11:40:56.753702 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:56.753754 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:56.754975 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:56.755991 (+ 1016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:56.755998 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:56.756344 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:56.756789 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:56.756811 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.758024 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.758026 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:56.759000 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:56.759009 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.759742 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.759744 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:56.759893 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:56.760502 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:56.760523 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:56.763099 (+ 2576us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":99,"threads_started":1}
W20260926 11:40:56.764809 15375 heartbeater.cc:644] Failed to heartbeat to 127.6.96.62:41081 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.96.62:41081: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Sep 26 11:40:56 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:40:56 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:40:56.770117 15378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:56.765231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.1:56081)
0926 11:40:56.765430 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:56.765442 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:56.765505 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:56.765724 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:56.765726 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:56.765853 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:56.765943 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:56.765948 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.766932 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.766934 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:56.767483 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:56.767490 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.767908 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.767911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:56.768000 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:56.768466 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:56.768476 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:56.769992 (+ 1516us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135}
W20260926 11:40:56.770224 15377 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:56.765348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:56081 (local address 127.6.96.62:41081)
0926 11:40:56.765470 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:56.765475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:56.765490 (+ 15us) server_negotiation.cc:407] Connection header received
0926 11:40:56.765574 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:56.765577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:56.765630 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:56.765720 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:56.766036 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.766812 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.767589 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.767815 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.767898 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:56.770128 (+ 2230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.1:56081: BlockingRecv error: recv got EOF from 127.6.96.1:56081 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 26 11:40:56 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:40:56 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:40:56.775524 15378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:56.770717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.1:52169)
0926 11:40:56.770896 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:56.770908 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:56.770966 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:56.771168 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:56.771171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:56.771279 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:56.771359 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:56.771363 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.772309 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.772311 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:56.772807 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:56.772812 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.773317 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.773319 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:56.773369 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:56.773777 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:56.773786 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:56.775408 (+ 1622us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":91}
W20260926 11:40:56.775630 15377 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:56.770822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:52169 (local address 127.6.96.62:41081)
0926 11:40:56.770945 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:56.770949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:56.770963 (+ 14us) server_negotiation.cc:407] Connection header received
0926 11:40:56.771039 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:56.771042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:56.771082 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:56.771159 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:56.771439 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.772213 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.772890 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.773221 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.773314 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:56.775513 (+ 2199us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.1:52169: BlockingRecv error: recv got EOF from 127.6.96.1:52169 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260926 11:40:56.775930 15375 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 11:40:56.872290 15379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:56.872577 15379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:56.872637 15379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:56.877054 15379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:56.877136 15379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:56.877219 15379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:40:56.883000 15379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:34403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.2.keytab
--principal=oryx/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:34741
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=36845
--tserver_master_addrs=127.6.96.62:41081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:56.884145 15379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:56.885185 15379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:56.894783 15384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:56.894783 15385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:56.894953 15379 server_base.cc:1034] running on GCE node
W20260926 11:40:56.894784 15387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:56.895489 15379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:56.896279 15379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:56.897462 15379 hybrid_clock.cc:648] HybridClock initialized: now 1790422856897429 us; error 52 us; skew 500 ppm
Sep 26 11:40:56 dist-test-slave-xxxx krb5kdc[14644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422856, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:56.900151 15379 init.cc:377] Logged in from keytab as oryx/127.6.96.2@KRBTEST.COM (short username oryx)
I20260926 11:40:56.901007 15379 webserver.cc:460] Webserver started at http://127.6.96.2:36845/ using document root <none> and password file <none>
I20260926 11:40:56.901546 15379 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:56.901613 15379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:56.905529 15379 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260926 11:40:56.907815 15394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:56.909192 15379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 11:40:56.909296 15379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "ca9f684414ba4c20a09436dab48e8bc3"
format_stamp: "Formatted at 2026-09-26 11:40:55 on dist-test-slave-xxxx"
server_key: "5725f37368283aada2282cf247d899e0"
server_key_iv: "c527c036cd5a358300d2e8392ca7a288"
server_key_version: "encryptionkey@0"
I20260926 11:40:56.909868 15379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:56.927841 15379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:56.930874 15379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:56.931039 15379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:56.931591 15379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:56.932410 15379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:56.932458 15379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:56.932498 15379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:56.932512 15379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:56.941006 15379 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:34741
I20260926 11:40:56.941041 15507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:34741 every 8 connection(s)
I20260926 11:40:56.941983 15379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:40:56.947429 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 15379
I20260926 11:40:56.947911 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:45911
--local_ip_for_outbound_sockets=127.6.96.3
--tserver_master_addrs=127.6.96.62:41081
--webserver_port=38305
--webserver_interface=127.6.96.3
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:34403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.96.3
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 11:40:56 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:40:56 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
W20260926 11:40:56.953578 15377 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:56.943823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:49087 (local address 127.6.96.62:41081)
0926 11:40:56.943984 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:56.943988 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:56.945027 (+ 1039us) server_negotiation.cc:407] Connection header received
0926 11:40:56.945875 (+ 848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:56.945878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:56.945929 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:56.946005 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:56.947219 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.948013 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.949624 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.949968 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.950093 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:56.953475 (+ 3382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.2:49087: BlockingRecv error: recv got EOF from 127.6.96.2:49087 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 11:40:56.954443 15510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:56.944217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.2:49087)
0926 11:40:56.944882 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:56.944924 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:56.945677 (+ 753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:56.946160 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:56.946167 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:56.946508 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:56.947066 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:56.947080 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.948174 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.948177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:56.949484 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:56.949495 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.950115 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.950118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:56.950255 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:56.950910 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:56.950927 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:56.953360 (+ 2433us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":108,"threads_started":1}
W20260926 11:40:56.955051 15508 heartbeater.cc:644] Failed to heartbeat to 127.6.96.62:41081 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.96.62:41081: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Sep 26 11:40:56 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:40:56 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:40:56.960675 15510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:56.955459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.2:52663)
0926 11:40:56.955611 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:56.955627 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:56.955713 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:56.955946 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:56.955949 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:56.956120 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:56.956255 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:56.956262 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.956950 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.956954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:56.957836 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:56.957849 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.958243 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.958248 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:56.958345 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:56.958789 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:56.958803 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:56.960577 (+ 1774us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":69}
W20260926 11:40:56.960855 15377 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:56.955580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:52663 (local address 127.6.96.62:41081)
0926 11:40:56.955674 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:56.955677 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:56.955689 (+ 12us) server_negotiation.cc:407] Connection header received
0926 11:40:56.955799 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:56.955801 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:56.955840 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:56.955902 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:56.956354 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.956847 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.957946 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.958143 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.958205 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:56.960737 (+ 2532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.2:52663: BlockingRecv error: recv got EOF from 127.6.96.2:52663 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Sep 26 11:40:56 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:40:56 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:40:56.965947 15510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:56.961258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.2:54039)
0926 11:40:56.961427 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:56.961440 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:56.961498 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:56.961732 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:56.961735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:56.961851 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:56.961923 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:56.961927 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.962914 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.962916 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:56.963422 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:56.963429 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.963926 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.963928 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:56.963980 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:56.964380 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:56.964391 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:56.965855 (+ 1464us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105}
W20260926 11:40:56.966131 15377 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:56.961354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:54039 (local address 127.6.96.62:41081)
0926 11:40:56.961458 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:56.961462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:56.961475 (+ 13us) server_negotiation.cc:407] Connection header received
0926 11:40:56.961570 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:56.961573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:56.961614 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:56.961708 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:56.962007 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.962804 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.963507 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:56.963840 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:56.963937 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:56.966021 (+ 2084us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.2:54039: BlockingRecv error: recv got EOF from 127.6.96.2:54039 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260926 11:40:56.966236 15508 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 11:40:57.061853 15511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:40:57.062112 15511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:40:57.062155 15511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:40:57.066207 15511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:40:57.066267 15511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:40:57.066331 15511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:40:57.071645 15511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:34403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/krb5kdc/oryx_127.6.96.3.keytab
--principal=oryx/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:45911
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=38305
--tserver_master_addrs=127.6.96.62:41081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:40:57.072681 15511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:40:57.073683 15511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:40:57.083391 15516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:40:57.083391 15517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:57.083642 15511 server_base.cc:1034] running on GCE node
W20260926 11:40:57.083411 15519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:40:57.084123 15511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:40:57.084770 15511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:40:57.085943 15511 hybrid_clock.cc:648] HybridClock initialized: now 1790422857085916 us; error 39 us; skew 500 ppm
Sep 26 11:40:57 dist-test-slave-xxxx krb5kdc[14644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422857, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:40:57.088814 15511 init.cc:377] Logged in from keytab as oryx/127.6.96.3@KRBTEST.COM (short username oryx)
I20260926 11:40:57.089763 15511 webserver.cc:460] Webserver started at http://127.6.96.3:38305/ using document root <none> and password file <none>
I20260926 11:40:57.090348 15511 fs_manager.cc:359] Metadata directory not provided
I20260926 11:40:57.090432 15511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:40:57.094439 15511 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:40:57.096807 15526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:57.098242 15511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 11:40:57.098383 15511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "e398b5ee88f44f66995f4215fe0f293f"
format_stamp: "Formatted at 2026-09-26 11:40:56 on dist-test-slave-xxxx"
server_key: "4cbe908e7dbf539c0eb5936078afddaf"
server_key_iv: "a479ed045578ef19fb4e90c557f31ba4"
server_key_version: "encryptionkey@0"
I20260926 11:40:57.099017 15511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:40:57.117053 15511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:40:57.120277 15511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:40:57.120447 15511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:40:57.121074 15511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:40:57.121899 15511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:40:57.121944 15511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:57.121982 15511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:40:57.122002 15511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:40:57.130242 15511 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:45911
I20260926 11:40:57.130266 15639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:45911 every 8 connection(s)
I20260926 11:40:57.131217 15511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:40:57.134516 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 15511
Sep 26 11:40:57 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:40:57 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
W20260926 11:40:57.141863 15377 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:57.132933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:35447 (local address 127.6.96.62:41081)
0926 11:40:57.133066 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:57.133071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:57.134113 (+ 1042us) server_negotiation.cc:407] Connection header received
0926 11:40:57.135151 (+ 1038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:57.135155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:57.135210 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:57.135290 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:57.136746 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:57.137244 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:57.138315 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:57.138515 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:57.138593 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:57.141715 (+ 3122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.3:35447: BlockingRecv error: recv got EOF from 127.6.96.3:35447 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 11:40:57.142477 15642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:57.133352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.3:35447)
0926 11:40:57.133971 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:57.134013 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:57.134950 (+ 937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:57.135491 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:57.135500 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:57.135946 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:57.136612 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:57.136633 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:57.137356 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:57.137359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:57.138206 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:57.138215 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:57.138641 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:57.138645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:57.138804 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:57.139315 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:57.139330 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:57.141587 (+ 2257us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":103,"threads_started":1}
W20260926 11:40:57.142954 15640 heartbeater.cc:644] Failed to heartbeat to 127.6.96.62:41081 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.96.62:41081: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
I20260926 11:40:57.143878 15644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:57.135520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51242 (local address 127.6.96.62:41081)
0926 11:40:57.135713 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:57.135716 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:57.135726 (+ 10us) server_negotiation.cc:407] Connection header received
0926 11:40:57.135757 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:57.135759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:57.135794 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:57.135866 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:57.136645 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:57.137173 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:57.138030 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:57.138250 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:57.138351 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:57.139747 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:40:57.139753 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:40:57.139765 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:40:57.139791 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:40:57.141575 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:57.142116 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:57.142123 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:57.142128 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:57.142286 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:40:57.142527 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:40:57.142530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:40:57.142532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:40:57.142971 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:40:57.143141 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0926 11:40:57.143515 (+ 374us) server_negotiation.cc:299] Negotiation successful
0926 11:40:57.143731 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":59,"threads_started":1}
Sep 26 11:40:57 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:40:57 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:40:57.148409 15642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:57.143354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.3:60567)
0926 11:40:57.143500 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:57.143512 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:57.143587 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:57.143825 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:57.143827 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:57.143954 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:57.144060 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:57.144066 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:57.144959 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:57.144962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:57.145723 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:57.145731 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:57.146168 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:57.146171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:57.146257 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:57.146687 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:57.146701 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:57.148324 (+ 1623us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89}
W20260926 11:40:57.148603 15377 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:57.143407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:60567 (local address 127.6.96.62:41081)
0926 11:40:57.143501 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:57.143505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:57.143551 (+ 46us) server_negotiation.cc:407] Connection header received
0926 11:40:57.143696 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:57.143700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:57.143741 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:57.143814 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:57.144168 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:57.144866 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:57.145829 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:57.146084 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:57.146169 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:57.148506 (+ 2337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.3:60567: BlockingRecv error: recv got EOF from 127.6.96.3:60567 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 26 11:40:57 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:40:57 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:40:57.153074 15642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:57.148905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.3:41809)
0926 11:40:57.149008 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:57.149022 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:57.149093 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:57.149293 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:57.149295 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:57.149420 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:57.149486 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:57.149491 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:57.150093 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:57.150095 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:57.150605 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:57.150623 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:57.151034 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:57.151037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:57.151094 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:57.151448 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:57.151459 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:57.152992 (+ 1533us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260926 11:40:57.153187 15377 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:57.148974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:41809 (local address 127.6.96.62:41081)
0926 11:40:57.149097 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:57.149100 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:57.149111 (+ 11us) server_negotiation.cc:407] Connection header received
0926 11:40:57.149156 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:57.149158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:57.149188 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:57.149244 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:57.149562 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:57.150014 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:57.150696 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:57.150939 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:57.151009 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:57.153088 (+ 2079us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.3:41809: BlockingRecv error: recv got EOF from 127.6.96.3:41809 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260926 11:40:57.153247 15640 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 26 11:40:57 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:40:57 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:40:57.781606 15645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:57.776497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.1:59405)
0926 11:40:57.776737 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:57.776749 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:57.776862 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:57.777082 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:57.777084 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:57.777247 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:57.777337 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:57.777342 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:57.778066 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:57.778069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:57.778709 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:57.778748 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:57.779229 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:57.779233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:57.779302 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:57.779780 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:57.779816 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:57.781485 (+ 1669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":68,"threads_started":1}
W20260926 11:40:57.781742 15646 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:57.776544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:59405 (local address 127.6.96.62:41081)
0926 11:40:57.776779 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:57.776783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:57.776795 (+ 12us) server_negotiation.cc:407] Connection header received
0926 11:40:57.776949 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:57.776952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:57.776996 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:57.777065 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:57.777421 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:57.777974 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:57.778853 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:57.779092 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:57.779168 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:57.781615 (+ 2447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.1:59405: BlockingRecv error: recv got EOF from 127.6.96.1:59405 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":89,"threads_started":1}
Sep 26 11:40:57 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:40:57 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:40:57.971890 15647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:57.966849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.2:35039)
0926 11:40:57.967111 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:57.967124 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:57.967220 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:57.967425 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:57.967427 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:57.967609 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:57.967705 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:57.967710 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:57.968428 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:57.968430 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:57.969018 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:57.969025 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:57.969464 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:57.969471 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:57.969529 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:57.969974 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:57.969984 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:57.971716 (+ 1732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":69,"threads_started":1}
W20260926 11:40:57.972003 15646 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:57.966973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:35039 (local address 127.6.96.62:41081)
0926 11:40:57.967096 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:57.967099 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:57.967177 (+ 78us) server_negotiation.cc:407] Connection header received
0926 11:40:57.967283 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:57.967285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:57.967328 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:57.967400 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:57.967809 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:57.968342 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:57.969141 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:57.969364 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:57.969460 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:57.971904 (+ 2444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.2:35039: BlockingRecv error: recv got EOF from 127.6.96.2:35039 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 26 11:40:58 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:40:58 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:40:58.172276 15648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:58.153857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.3:57413)
0926 11:40:58.154107 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:58.154120 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:58.154218 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:58.154434 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:58.154435 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:58.154602 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:58.154703 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:58.154708 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:58.155419 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:58.155421 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:58.156003 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:58.156011 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:58.156449 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:58.156451 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:58.156510 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:58.170124 (+ 13614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:58.170140 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:58.172111 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":67,"threads_started":1}
W20260926 11:40:58.172379 15646 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:58.153937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:57413 (local address 127.6.96.62:41081)
0926 11:40:58.154069 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:58.154073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:58.154183 (+ 110us) server_negotiation.cc:407] Connection header received
0926 11:40:58.154292 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:58.154295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:58.154340 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:58.154408 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:58.154793 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:58.155330 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:58.156129 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:58.156353 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:58.156434 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:58.172260 (+ 15826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.3:57413: BlockingRecv error: recv got EOF from 127.6.96.3:57413 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 26 11:40:58 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:40:58 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:40:58.787398 15649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:58.782267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.1:60861)
0926 11:40:58.782540 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:58.782553 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:58.782645 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:58.782909 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:58.782912 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:58.783080 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:58.783177 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:58.783182 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:58.783932 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:58.783935 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:58.784504 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:58.784512 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:58.784919 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:58.784922 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:58.784989 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:58.785514 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:58.785526 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:58.787259 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":80,"threads_started":1}
W20260926 11:40:58.787556 15650 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:58.782359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:60861 (local address 127.6.96.62:41081)
0926 11:40:58.782587 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:58.782590 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:58.782603 (+ 13us) server_negotiation.cc:407] Connection header received
0926 11:40:58.782722 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:58.782725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:58.782769 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:58.782861 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:58.783289 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:58.783814 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:58.784604 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:58.784822 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:58.784895 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:58.787409 (+ 2514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.1:60861: BlockingRecv error: recv got EOF from 127.6.96.1:60861 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":79,"threads_started":1}
Sep 26 11:40:58 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:40:58 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:40:58.978152 15651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:58.972695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.2:53437)
0926 11:40:58.972967 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:58.972982 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:58.973081 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:58.973311 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:58.973313 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:58.973499 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:58.973595 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:58.973601 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:58.974326 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:58.974329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:58.974996 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:58.975008 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:58.975438 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:58.975440 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:58.975532 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:58.975987 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:58.976003 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:58.977970 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":82,"threads_started":1}
W20260926 11:40:58.978196 15650 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:58.972844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:53437 (local address 127.6.96.62:41081)
0926 11:40:58.972983 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:58.972987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:58.973055 (+ 68us) server_negotiation.cc:407] Connection header received
0926 11:40:58.973151 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:58.973153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:58.973197 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:58.973280 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:58.973690 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:58.974223 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:58.975107 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:58.975339 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:58.975413 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:58.978099 (+ 2686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.2:53437: BlockingRecv error: recv got EOF from 127.6.96.2:53437 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 26 11:40:59 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:40:59 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:40:59.178200 15652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:59.173073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.3:35097)
0926 11:40:59.173316 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:59.173329 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:59.173411 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:59.173616 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:59.173618 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:59.173780 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:59.173871 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:59.173875 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:59.174629 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:59.174632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:59.175263 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:59.175272 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:59.175736 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:59.175741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:59.175821 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:59.176307 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:59.176319 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:59.178018 (+ 1699us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":75,"threads_started":1}
W20260926 11:40:59.178251 15650 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:59.173239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:35097 (local address 127.6.96.62:41081)
0926 11:40:59.173353 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:59.173356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:59.173370 (+ 14us) server_negotiation.cc:407] Connection header received
0926 11:40:59.173478 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:59.173480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:59.173524 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:59.173595 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:59.173958 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:59.174517 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:59.175371 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:59.175611 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:59.175697 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:59.178152 (+ 2455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.3:35097: BlockingRecv error: recv got EOF from 127.6.96.3:35097 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 26 11:40:59 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:40:59 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:40:59.793067 15653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:59.788134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.1:39051)
0926 11:40:59.788410 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:59.788426 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:59.788518 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:59.788743 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:59.788745 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:59.788915 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:59.789009 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:59.789014 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:59.789701 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:59.789703 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:59.790241 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:59.790247 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:59.790634 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:59.790636 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:59.790698 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:59.791214 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:59.791225 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:59.792927 (+ 1702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":68,"threads_started":1}
W20260926 11:40:59.793190 15654 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:59.788243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:39051 (local address 127.6.96.62:41081)
0926 11:40:59.788527 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:59.788531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:59.788564 (+ 33us) server_negotiation.cc:407] Connection header received
0926 11:40:59.788597 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:59.788599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:59.788640 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:59.788705 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:59.789101 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:59.789625 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:59.790326 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:59.790550 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:59.790620 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:59.793073 (+ 2453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.1:39051: BlockingRecv error: recv got EOF from 127.6.96.1:39051 (error 108)
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":101,"threads_started":1}
Sep 26 11:40:59 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:40:59 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:40:59.983929 15655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:40:59.978885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.2:33827)
0926 11:40:59.979117 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0926 11:40:59.979129 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:40:59.979238 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:40:59.979443 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:40:59.979445 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:40:59.979609 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:40:59.979706 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:59.979712 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:59.980387 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:59.980390 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:59.980935 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:40:59.980942 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:40:59.981300 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:40:59.981302 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:40:59.981362 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:59.981804 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:40:59.981818 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:40:59.983791 (+ 1973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":74,"threads_started":1}
W20260926 11:40:59.984019 15654 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:40:59.978936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:33827 (local address 127.6.96.62:41081)
0926 11:40:59.979063 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 11:40:59.979067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:40:59.979194 (+ 127us) server_negotiation.cc:407] Connection header received
0926 11:40:59.979307 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:40:59.979309 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:40:59.979354 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:40:59.979422 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:40:59.979784 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:59.980309 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:59.981020 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:40:59.981225 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:40:59.981291 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:40:59.983927 (+ 2636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.2:33827: BlockingRecv error: recv got EOF from 127.6.96.2:33827 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 26 11:41:00 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:00 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:00.184399 15656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:00.179042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.3:33507)
0926 11:41:00.179431 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:00.179443 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:00.179550 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:00.179777 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:00.179780 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:00.179972 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:00.180067 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:00.180073 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:00.180809 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:00.180812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:00.181363 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:00.181378 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:00.181803 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:00.181807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:00.181889 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:00.182364 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:00.182379 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:00.184261 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":77,"threads_started":1}
W20260926 11:41:00.184517 15654 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:00.179078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:33507 (local address 127.6.96.62:41081)
0926 11:41:00.179210 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:00.179214 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:00.179516 (+ 302us) server_negotiation.cc:407] Connection header received
0926 11:41:00.179631 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:00.179634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:00.179684 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:00.179743 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:00.180180 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:00.180700 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:00.181486 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:00.181707 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:00.181788 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:00.184401 (+ 2613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.3:33507: BlockingRecv error: recv got EOF from 127.6.96.3:33507 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 26 11:41:00 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:00 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:00.799018 15657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:00.793801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.1:38345)
0926 11:41:00.794056 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:00.794070 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:00.794160 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:00.794384 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:00.794386 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:00.794585 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:00.794696 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:00.794704 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:00.795505 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:00.795507 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:00.796108 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:00.796117 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:00.796540 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:00.796544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:00.796617 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:00.797102 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:00.797114 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:00.798900 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":115,"threads_started":1}
W20260926 11:41:00.799190 15658 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:00.793848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:38345 (local address 127.6.96.62:41081)
0926 11:41:00.794084 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:00.794087 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:00.794114 (+ 27us) server_negotiation.cc:407] Connection header received
0926 11:41:00.794240 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:00.794242 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:00.794284 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:00.794358 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:00.794803 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:00.795417 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:00.796217 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:00.796443 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:00.796527 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:00.799056 (+ 2529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.1:38345: BlockingRecv error: recv got EOF from 127.6.96.1:38345 (error 108)
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":65,"threads_started":1}
Sep 26 11:41:00 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:00 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:00.989655 15659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:00.984741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.2:47699)
0926 11:41:00.984987 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:00.985008 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:00.985114 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:00.985376 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:00.985378 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:00.985551 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:00.985643 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:00.985648 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:00.986376 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:00.986379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:00.986984 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:00.986993 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:00.987365 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:00.987368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:00.987423 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:00.987889 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:00.987900 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:00.989537 (+ 1637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":69,"threads_started":1}
W20260926 11:41:00.989807 15658 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:00.984855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:47699 (local address 127.6.96.62:41081)
0926 11:41:00.984992 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:00.984997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:00.985070 (+ 73us) server_negotiation.cc:407] Connection header received
0926 11:41:00.985228 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:00.985231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:00.985281 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:00.985352 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:00.985739 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:00.986290 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:00.987075 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:00.987290 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:00.987357 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:00.989679 (+ 2322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.2:47699: BlockingRecv error: recv got EOF from 127.6.96.2:47699 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 26 11:41:01 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:01 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:01.190284 15660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:01.185096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.3:48387)
0926 11:41:01.185397 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:01.185412 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:01.185537 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:01.185791 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:01.185794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:01.185959 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:01.186073 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:01.186079 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:01.186758 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:01.186760 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:01.187349 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:01.187356 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:01.187718 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:01.187720 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:01.187775 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:01.188212 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:01.188222 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:01.190171 (+ 1949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":100,"threads_started":1}
W20260926 11:41:01.190392 15658 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:01.185209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:48387 (local address 127.6.96.62:41081)
0926 11:41:01.185397 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:01.185401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:01.185474 (+ 73us) server_negotiation.cc:407] Connection header received
0926 11:41:01.185628 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:01.185632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:01.185676 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:01.185758 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:01.186168 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:01.186678 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:01.187432 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:01.187643 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:01.187708 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:01.190293 (+ 2585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.3:48387: BlockingRecv error: recv got EOF from 127.6.96.3:48387 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Sep 26 11:41:01 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:01 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:01.805136 15661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:01.799813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.1:46181)
0926 11:41:01.800095 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:01.800110 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:01.800207 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:01.800429 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:01.800431 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:01.800612 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:01.800711 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:01.800715 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:01.801456 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:01.801458 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:01.802075 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:01.802084 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:01.802502 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:01.802506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:01.802590 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:01.803168 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:01.803183 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:01.805015 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":86,"threads_started":1}
W20260926 11:41:01.805274 15662 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:01.799890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:46181 (local address 127.6.96.62:41081)
0926 11:41:01.800129 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:01.800133 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:01.800163 (+ 30us) server_negotiation.cc:407] Connection header received
0926 11:41:01.800278 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:01.800281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:01.800330 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:01.800437 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:01.800819 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:01.801371 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:01.802162 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:01.802382 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:01.802483 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:01.805141 (+ 2658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.1:46181: BlockingRecv error: recv got EOF from 127.6.96.1:46181 (error 108)
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":75,"threads_started":1}
Sep 26 11:41:01 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:01 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:01.995735 15663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:01.990482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.2:58653)
0926 11:41:01.990756 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:01.990769 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:01.990903 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:01.991216 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:01.991219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:01.991396 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:01.991494 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:01.991501 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:01.992234 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:01.992236 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:01.992820 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:01.992829 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:01.993216 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:01.993218 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:01.993276 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:01.993726 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:01.993738 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:01.995587 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":90,"threads_started":1}
W20260926 11:41:01.995818 15662 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:01.990665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:58653 (local address 127.6.96.62:41081)
0926 11:41:01.990781 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:01.990784 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:01.990862 (+ 78us) server_negotiation.cc:407] Connection header received
0926 11:41:01.991003 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:01.991006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:01.991062 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:01.991141 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:01.991612 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:01.992128 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:01.992912 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:01.993133 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:01.993206 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:01.995721 (+ 2515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.2:58653: BlockingRecv error: recv got EOF from 127.6.96.2:58653 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 26 11:41:02 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:02 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:02.196214 15664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:02.191019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.3:38505)
0926 11:41:02.191277 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:02.191290 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:02.191389 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:02.191676 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:02.191679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:02.191855 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:02.191953 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:02.191958 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:02.192658 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:02.192661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:02.193250 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:02.193263 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:02.193619 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:02.193622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:02.193682 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:02.194137 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:02.194148 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:02.196053 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":102,"threads_started":1}
W20260926 11:41:02.196269 15662 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:02.191101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:38505 (local address 127.6.96.62:41081)
0926 11:41:02.191238 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:02.191242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:02.191350 (+ 108us) server_negotiation.cc:407] Connection header received
0926 11:41:02.191459 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:02.191461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:02.191517 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:02.191584 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:02.192052 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:02.192561 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:02.193339 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:02.193543 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:02.193608 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:02.196168 (+ 2560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.3:38505: BlockingRecv error: recv got EOF from 127.6.96.3:38505 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 26 11:41:02 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:02 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:02.811326 15665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:02.805868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.1:52197)
0926 11:41:02.806125 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:02.806141 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:02.806246 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:02.806468 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:02.806470 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:02.806643 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:02.806738 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:02.806743 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:02.807560 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:02.807563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:02.808145 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:02.808154 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:02.808709 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:02.808713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:02.808807 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:02.809348 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:02.809363 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:02.811211 (+ 1848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":74,"threads_started":1}
W20260926 11:41:02.811482 15666 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:02.806004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:52197 (local address 127.6.96.62:41081)
0926 11:41:02.806236 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:02.806240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:02.806253 (+ 13us) server_negotiation.cc:407] Connection header received
0926 11:41:02.806321 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:02.806323 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:02.806365 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:02.806440 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:02.806891 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:02.807448 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:02.808275 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:02.808577 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:02.808673 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:02.811344 (+ 2671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.1:52197: BlockingRecv error: recv got EOF from 127.6.96.1:52197 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":71,"threads_started":1}
Sep 26 11:41:02 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:02 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:03.001483 15667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:02.996548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.2:53337)
0926 11:41:02.996797 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:02.996809 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:02.996913 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:02.997148 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:02.997150 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:02.997331 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:02.997421 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:02.997426 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:02.998159 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:02.998163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:02.998720 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:02.998728 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:02.999183 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:02.999186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:02.999267 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:02.999710 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:02.999721 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:03.001336 (+ 1615us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":81,"threads_started":1}
W20260926 11:41:03.001588 15666 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:02.996608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:53337 (local address 127.6.96.62:41081)
0926 11:41:02.996747 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:02.996752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:02.996874 (+ 122us) server_negotiation.cc:407] Connection header received
0926 11:41:02.996994 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:02.996997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:02.997046 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:02.997112 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:02.997546 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:02.998060 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:02.998845 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:02.999084 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:02.999171 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:03.001470 (+ 2299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.2:53337: BlockingRecv error: recv got EOF from 127.6.96.2:53337 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 26 11:41:03 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:03 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:03.208940 15668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:03.197001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.3:47963)
0926 11:41:03.197251 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:03.197266 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:03.197361 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:03.197587 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:03.197589 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:03.197762 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:03.197858 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:03.197863 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:03.198584 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:03.198587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:03.199221 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:03.199233 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:03.199694 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:03.199697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:03.199769 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:03.207027 (+ 7258us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:03.207041 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:03.208821 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":83,"threads_started":1}
W20260926 11:41:03.209139 15666 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:03.197046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:47963 (local address 127.6.96.62:41081)
0926 11:41:03.197171 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:03.197175 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:03.197325 (+ 150us) server_negotiation.cc:407] Connection header received
0926 11:41:03.197444 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:03.197447 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:03.197490 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:03.197559 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:03.197943 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:03.198483 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:03.199345 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:03.199567 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:03.199646 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:03.209021 (+ 9375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.3:47963: BlockingRecv error: recv got EOF from 127.6.96.3:47963 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 26 11:41:03 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:03 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:03.817444 15669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:03.812090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.1:46975)
0926 11:41:03.812385 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:03.812400 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:03.812501 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:03.812723 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:03.812725 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:03.812888 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:03.812985 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:03.812990 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:03.813827 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:03.813832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:03.814452 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:03.814467 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:03.815026 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:03.815030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:03.815121 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:03.815609 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:03.815621 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:03.817297 (+ 1676us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":69,"threads_started":1}
W20260926 11:41:03.817541 15670 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:03.812183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:46975 (local address 127.6.96.62:41081)
0926 11:41:03.812390 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:03.812394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:03.812462 (+ 68us) server_negotiation.cc:407] Connection header received
0926 11:41:03.812574 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:03.812576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:03.812619 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:03.812702 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:03.813120 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:03.813696 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:03.814593 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:03.814910 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:03.815005 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:03.817428 (+ 2423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.1:46975: BlockingRecv error: recv got EOF from 127.6.96.1:46975 (error 108)
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":63,"threads_started":1}
Sep 26 11:41:04 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:04 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:04.008080 15671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:04.002274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.2:34267)
0926 11:41:04.002669 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:04.002682 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:04.002789 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:04.003109 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:04.003112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:04.003312 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:04.003404 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:04.003409 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:04.004184 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:04.004188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:04.004928 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:04.004939 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:04.005439 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:04.005443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:04.005521 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:04.006020 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:04.006059 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:04.007913 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":96,"threads_started":1}
W20260926 11:41:04.008189 15670 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:04.002360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:34267 (local address 127.6.96.62:41081)
0926 11:41:04.002533 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:04.002538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:04.002772 (+ 234us) server_negotiation.cc:407] Connection header received
0926 11:41:04.002908 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:04.002911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:04.002968 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:04.003055 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:04.003516 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:04.004060 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:04.005066 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:04.005330 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:04.005421 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:04.008087 (+ 2666us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.2:34267: BlockingRecv error: recv got EOF from 127.6.96.2:34267 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Sep 26 11:41:04 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:04 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:04.215190 15672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:04.209741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.3:38609)
0926 11:41:04.210051 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:04.210065 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:04.210175 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:04.210433 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:04.210436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:04.210611 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:04.210706 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:04.210712 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:04.211530 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:04.211533 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:04.212207 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:04.212218 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:04.212658 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:04.212662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:04.212743 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:04.213226 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:04.213237 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:04.215070 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":67,"threads_started":1}
W20260926 11:41:04.215327 15670 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:04.209844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:38609 (local address 127.6.96.62:41081)
0926 11:41:04.209992 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:04.209998 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:04.210154 (+ 156us) server_negotiation.cc:407] Connection header received
0926 11:41:04.210244 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:04.210246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:04.210292 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:04.210375 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:04.210794 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:04.211384 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:04.212319 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:04.212536 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:04.212642 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:04.215218 (+ 2576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.3:38609: BlockingRecv error: recv got EOF from 127.6.96.3:38609 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 26 11:41:04 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:04 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:04.824165 15673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:04.818222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.1:47275)
0926 11:41:04.818512 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:04.818528 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:04.818652 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:04.818936 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:04.818938 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:04.819152 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:04.819272 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:04.819280 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:04.820350 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:04.820354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:04.821255 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:04.821267 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:04.821704 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:04.821707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:04.821768 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:04.822269 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:04.822284 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:04.823983 (+ 1699us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":74,"threads_started":1}
W20260926 11:41:04.824249 15674 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:04.818375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:47275 (local address 127.6.96.62:41081)
0926 11:41:04.818596 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:04.818599 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:04.818632 (+ 33us) server_negotiation.cc:407] Connection header received
0926 11:41:04.818739 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:04.818741 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:04.818786 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:04.818912 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:04.819398 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:04.820240 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:04.821382 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:04.821601 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:04.821680 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:04.824130 (+ 2450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.1:47275: BlockingRecv error: recv got EOF from 127.6.96.1:47275 (error 108)
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":72,"threads_started":1}
Sep 26 11:41:05 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:05 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:05.014017 15675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:05.008895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.2:42783)
0926 11:41:05.009130 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:05.009143 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:05.009238 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:05.009491 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:05.009494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:05.009666 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:05.009762 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:05.009767 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:05.010622 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:05.010625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:05.011255 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:05.011264 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:05.011673 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:05.011675 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:05.011732 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:05.012208 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:05.012219 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:05.013903 (+ 1684us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":67,"threads_started":1}
W20260926 11:41:05.014120 15674 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:05.008965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:42783 (local address 127.6.96.62:41081)
0926 11:41:05.009134 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:05.009137 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:05.009202 (+ 65us) server_negotiation.cc:407] Connection header received
0926 11:41:05.009304 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:05.009306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:05.009350 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:05.009418 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:05.009861 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:05.010518 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:05.011358 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:05.011573 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:05.011666 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:05.014023 (+ 2357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.2:42783: BlockingRecv error: recv got EOF from 127.6.96.2:42783 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Sep 26 11:41:05 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:05 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:05.221758 15676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:05.216046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.3:56757)
0926 11:41:05.216322 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:05.216337 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:05.216483 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:05.216809 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:05.216813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:05.216991 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:05.217136 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:05.217143 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:05.217992 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:05.217995 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:05.218605 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:05.218613 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:05.219220 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:05.219223 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:05.219293 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:05.219776 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:05.219787 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:05.221637 (+ 1850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":78,"threads_started":1}
W20260926 11:41:05.221935 15674 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:05.216103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:56757 (local address 127.6.96.62:41081)
0926 11:41:05.216259 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:05.216263 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:05.216455 (+ 192us) server_negotiation.cc:407] Connection header received
0926 11:41:05.216583 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:05.216586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:05.216646 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:05.216727 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:05.217261 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:05.217882 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:05.218725 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:05.219092 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:05.219201 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:05.221799 (+ 2598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.3:56757: BlockingRecv error: recv got EOF from 127.6.96.3:56757 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 26 11:41:05 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:05 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:05.830773 15677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:05.824972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.1:39963)
0926 11:41:05.825452 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:05.825468 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:05.825608 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:05.825930 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:05.825934 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:05.826142 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:05.826246 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:05.826252 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:05.827111 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:05.827115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:05.827707 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:05.827717 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:05.828176 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:05.828180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:05.828272 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:05.828770 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:05.828782 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:05.830650 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":85,"threads_started":1}
W20260926 11:41:05.831027 15678 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:05.825042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:39963 (local address 127.6.96.62:41081)
0926 11:41:05.825304 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:05.825309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:05.825567 (+ 258us) server_negotiation.cc:407] Connection header received
0926 11:41:05.825710 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:05.825714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:05.825769 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:05.825865 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:05.826358 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:05.826999 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:05.827821 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:05.828046 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:05.828135 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:05.830873 (+ 2738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.1:39963: BlockingRecv error: recv got EOF from 127.6.96.1:39963 (error 108)
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":103,"threads_started":1}
Sep 26 11:41:06 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:06 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:06.019726 15679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:06.014662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.2:48071)
0926 11:41:06.014934 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:06.014947 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:06.015047 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:06.015298 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:06.015300 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:06.015466 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:06.015557 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:06.015562 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:06.016265 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:06.016267 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:06.016817 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:06.016824 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:06.017273 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:06.017275 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:06.017335 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:06.017807 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:06.017818 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:06.019594 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":81,"threads_started":1}
W20260926 11:41:06.019816 15678 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:06.014741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:48071 (local address 127.6.96.62:41081)
0926 11:41:06.014912 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:06.014916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:06.015004 (+ 88us) server_negotiation.cc:407] Connection header received
0926 11:41:06.015117 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:06.015119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:06.015187 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:06.015254 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:06.015635 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:06.016168 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:06.016927 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:06.017169 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:06.017249 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:06.019712 (+ 2463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.2:48071: BlockingRecv error: recv got EOF from 127.6.96.2:48071 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 26 11:41:06 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:06 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:06.227789 15680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:06.222478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.3:51979)
0926 11:41:06.222741 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:06.222754 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:06.222870 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:06.223090 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:06.223092 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:06.223273 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:06.223368 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:06.223373 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:06.224148 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:06.224151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:06.224719 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:06.224729 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:06.225149 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:06.225152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:06.225216 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:06.225824 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:06.225838 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:06.227669 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":77,"threads_started":1}
W20260926 11:41:06.227912 15678 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:06.222572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:51979 (local address 127.6.96.62:41081)
0926 11:41:06.222707 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:06.222711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:06.222812 (+ 101us) server_negotiation.cc:407] Connection header received
0926 11:41:06.222957 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:06.222959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:06.223004 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:06.223069 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:06.223485 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:06.224028 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:06.224821 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:06.225046 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:06.225130 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:06.227794 (+ 2664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.3:51979: BlockingRecv error: recv got EOF from 127.6.96.3:51979 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 26 11:41:06 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:06 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:06.837246 15681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:06.831683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.1:37939)
0926 11:41:06.831991 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:06.832005 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:06.832112 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:06.832418 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:06.832421 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:06.832595 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:06.832713 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:06.832719 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:06.833486 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:06.833489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:06.834128 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:06.834141 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:06.834662 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:06.834664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:06.834740 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:06.835363 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:06.835375 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:06.837130 (+ 1755us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":88,"threads_started":1}
W20260926 11:41:06.837419 15682 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:06.831810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:37939 (local address 127.6.96.62:41081)
0926 11:41:06.832033 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:06.832036 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:06.832070 (+ 34us) server_negotiation.cc:407] Connection header received
0926 11:41:06.832242 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:06.832246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:06.832296 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:06.832383 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:06.832822 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:06.833360 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:06.834256 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:06.834554 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:06.834653 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:06.837292 (+ 2639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.1:37939: BlockingRecv error: recv got EOF from 127.6.96.1:37939 (error 108)
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":58,"threads_started":1}
Sep 26 11:41:07 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:07 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:07.025784 15683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:07.020559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.2:50927)
0926 11:41:07.020806 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:07.020820 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:07.020932 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:07.021191 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:07.021193 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:07.021403 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:07.021512 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:07.021518 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:07.022385 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:07.022387 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:07.023017 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:07.023026 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:07.023458 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:07.023461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:07.023525 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:07.024009 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:07.024021 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:07.025656 (+ 1635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":75,"threads_started":1}
W20260926 11:41:07.025890 15682 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:07.020617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:50927 (local address 127.6.96.62:41081)
0926 11:41:07.020748 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:07.020752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:07.020900 (+ 148us) server_negotiation.cc:407] Connection header received
0926 11:41:07.021017 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:07.021020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:07.021072 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:07.021156 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:07.021646 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:07.022284 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:07.023149 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:07.023369 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:07.023450 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:07.025789 (+ 2339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.2:50927: BlockingRecv error: recv got EOF from 127.6.96.2:50927 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 26 11:41:07 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:07 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:07.234293 15684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:07.228673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.3:40105)
0926 11:41:07.228949 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:07.228966 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:07.229076 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:07.229359 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:07.229362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:07.229561 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:07.229697 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:07.229705 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:07.230579 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:07.230583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:07.231350 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:07.231360 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:07.231778 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:07.231781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:07.231856 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:07.232343 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:07.232357 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:07.234176 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":93,"threads_started":1}
W20260926 11:41:07.234462 15682 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:07.228860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:40105 (local address 127.6.96.62:41081)
0926 11:41:07.228994 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:07.228998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:07.229017 (+ 19us) server_negotiation.cc:407] Connection header received
0926 11:41:07.229210 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:07.229213 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:07.229257 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:07.229323 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:07.229817 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:07.230481 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:07.231453 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:07.231683 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:07.231766 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:07.234341 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.3:40105: BlockingRecv error: recv got EOF from 127.6.96.3:40105 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 26 11:41:07 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:07 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:07.843607 15685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:07.838083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.1:43693)
0926 11:41:07.838333 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:07.838347 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:07.838440 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:07.838799 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:07.838801 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:07.839023 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:07.839137 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:07.839143 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:07.840081 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:07.840084 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:07.840786 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:07.840796 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:07.841205 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:07.841207 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:07.841267 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:07.841769 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:07.841781 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:07.843471 (+ 1690us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":72,"threads_started":1}
W20260926 11:41:07.843777 15686 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:07.838232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:43693 (local address 127.6.96.62:41081)
0926 11:41:07.838516 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:07.838519 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:07.838535 (+ 16us) server_negotiation.cc:407] Connection header received
0926 11:41:07.838576 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:07.838578 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:07.838620 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:07.838722 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:07.839257 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:07.839987 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:07.840893 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:07.841116 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:07.841191 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:07.843615 (+ 2424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.1:43693: BlockingRecv error: recv got EOF from 127.6.96.1:43693 (error 108)
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":95,"threads_started":1}
Sep 26 11:41:08 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:08 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:08.032001 15687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:08.026542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.2:39893)
0926 11:41:08.026775 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:08.026795 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:08.026931 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:08.027203 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:08.027206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:08.027392 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:08.027493 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:08.027499 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:08.028229 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:08.028234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:08.028837 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:08.028848 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:08.029346 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:08.029351 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:08.029425 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:08.029910 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:08.029923 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:08.031824 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":66,"threads_started":1}
W20260926 11:41:08.032068 15686 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:08.026619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:39893 (local address 127.6.96.62:41081)
0926 11:41:08.026765 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:08.026769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:08.026872 (+ 103us) server_negotiation.cc:407] Connection header received
0926 11:41:08.027023 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:08.027026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:08.027080 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:08.027161 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:08.027586 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:08.028100 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:08.028979 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:08.029211 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:08.029296 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:08.031966 (+ 2670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.2:39893: BlockingRecv error: recv got EOF from 127.6.96.2:39893 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 26 11:41:08 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:08 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:08.245764 15688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:08.235104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.3:52723)
0926 11:41:08.235359 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:08.235376 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:08.235482 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:08.235780 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:08.235782 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:08.235964 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:08.236059 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:08.236064 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:08.237023 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:08.237027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:08.237761 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:08.237772 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:08.238165 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:08.238168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:08.238224 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:08.243789 (+ 5565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:08.243803 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:08.245620 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":84,"threads_started":1}
W20260926 11:41:08.245882 15686 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:08.235182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:52723 (local address 127.6.96.62:41081)
0926 11:41:08.235335 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:08.235339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:08.235443 (+ 104us) server_negotiation.cc:407] Connection header received
0926 11:41:08.235619 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:08.235622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:08.235673 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:08.235754 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:08.236189 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:08.236914 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:08.237872 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:08.238083 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:08.238158 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:08.245777 (+ 7619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.3:52723: BlockingRecv error: recv got EOF from 127.6.96.3:52723 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 26 11:41:08 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:08 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:08.849641 15689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:08.844399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.1:36521)
0926 11:41:08.844681 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:08.844696 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:08.844780 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:08.844987 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:08.844988 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:08.845156 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:08.845250 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:08.845255 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:08.846169 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:08.846173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:08.846858 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:08.846871 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:08.847283 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:08.847285 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:08.847365 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:08.847804 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:08.847816 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:08.849494 (+ 1678us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":82,"threads_started":1}
W20260926 11:41:08.849740 15690 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:08.844484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:36521 (local address 127.6.96.62:41081)
0926 11:41:08.844721 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:08.844724 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:08.844757 (+ 33us) server_negotiation.cc:407] Connection header received
0926 11:41:08.844861 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:08.844863 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:08.844906 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:08.844972 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:08.845368 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:08.846040 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:08.846970 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:08.847195 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:08.847269 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:08.849627 (+ 2358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.1:36521: BlockingRecv error: recv got EOF from 127.6.96.1:36521 (error 108)
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":54,"threads_started":1}
Sep 26 11:41:09 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:09 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:09.037544 15691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:09.032695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.2:60075)
0926 11:41:09.032945 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:09.032958 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:09.033048 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:09.033296 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:09.033299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:09.033478 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:09.033575 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:09.033582 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:09.034289 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:09.034292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:09.034890 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:09.034901 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:09.035282 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:09.035284 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:09.035348 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:09.035825 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:09.035836 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:09.037428 (+ 1592us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":87,"threads_started":1}
W20260926 11:41:09.037652 15690 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:09.032764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:60075 (local address 127.6.96.62:41081)
0926 11:41:09.032886 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:09.032890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:09.033014 (+ 124us) server_negotiation.cc:407] Connection header received
0926 11:41:09.033113 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:09.033115 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:09.033155 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:09.033219 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:09.033668 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:09.034187 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:09.034991 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:09.035206 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:09.035274 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:09.037554 (+ 2280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.2:60075: BlockingRecv error: recv got EOF from 127.6.96.2:60075 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 26 11:41:09 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:09 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:09.251328 15692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:09.246482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.3:46541)
0926 11:41:09.246761 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:09.246773 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:09.246890 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:09.247081 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:09.247083 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:09.247258 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:09.247358 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:09.247364 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:09.248042 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:09.248044 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:09.248607 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:09.248615 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:09.248981 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:09.248984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:09.249042 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:09.249530 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:09.249541 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:09.251207 (+ 1666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":83,"threads_started":1}
W20260926 11:41:09.251453 15690 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:09.246573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:46541 (local address 127.6.96.62:41081)
0926 11:41:09.246692 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:09.246696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:09.246845 (+ 149us) server_negotiation.cc:407] Connection header received
0926 11:41:09.246957 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:09.246959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:09.247006 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:09.247070 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:09.247437 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:09.247959 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:09.248689 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:09.248905 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:09.248970 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:09.251335 (+ 2365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.3:46541: BlockingRecv error: recv got EOF from 127.6.96.3:46541 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 26 11:41:09 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:09 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:09.855508 15693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:09.850441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.1:46421)
0926 11:41:09.850717 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:09.850732 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:09.850849 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:09.851140 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:09.851142 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:09.851331 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:09.851428 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:09.851433 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:09.852134 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:09.852136 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:09.852694 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:09.852701 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:09.853112 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:09.853115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:09.853185 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:09.853667 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:09.853678 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:09.855386 (+ 1708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":87,"threads_started":1}
W20260926 11:41:09.855636 15694 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:09.850579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:46421 (local address 127.6.96.62:41081)
0926 11:41:09.850796 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:09.850799 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:09.850813 (+ 14us) server_negotiation.cc:407] Connection header received
0926 11:41:09.850973 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:09.850976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:09.851024 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:09.851103 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:09.851520 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:09.852045 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:09.852777 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:09.852999 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:09.853067 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:09.855521 (+ 2454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.1:46421: BlockingRecv error: recv got EOF from 127.6.96.1:46421 (error 108)
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":55,"threads_started":1}
Sep 26 11:41:10 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:10 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:10.043339 15695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:10.038251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.2:60345)
0926 11:41:10.038506 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:10.038519 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:10.038608 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:10.038820 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:10.038846 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:10.039025 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:10.039118 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:10.039123 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:10.039835 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:10.039838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:10.040405 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:10.040413 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:10.040803 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:10.040806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:10.040869 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:10.041324 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:10.041336 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:10.043183 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":78,"threads_started":1}
W20260926 11:41:10.043459 15694 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:10.038442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:60345 (local address 127.6.96.62:41081)
0926 11:41:10.038569 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:10.038572 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:10.038585 (+ 13us) server_negotiation.cc:407] Connection header received
0926 11:41:10.038682 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:10.038685 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:10.038726 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:10.038790 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:10.039201 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:10.039736 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:10.040495 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:10.040706 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:10.040777 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:10.043338 (+ 2561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.2:60345: BlockingRecv error: recv got EOF from 127.6.96.2:60345 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 26 11:41:10 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:10 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:10.257081 15696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:10.252050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.3:37277)
0926 11:41:10.252300 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:10.252314 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:10.252414 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:10.252715 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:10.252717 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:10.252887 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:10.252992 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:10.252998 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:10.253744 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:10.253747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:10.254306 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:10.254314 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:10.254674 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:10.254677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:10.254734 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:10.255264 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:10.255275 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:10.256961 (+ 1686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":74,"threads_started":1}
W20260926 11:41:10.257212 15694 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:10.252112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:37277 (local address 127.6.96.62:41081)
0926 11:41:10.252229 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:10.252233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:10.252484 (+ 251us) server_negotiation.cc:407] Connection header received
0926 11:41:10.252562 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:10.252564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:10.252612 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:10.252672 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:10.253110 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:10.253647 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:10.254392 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:10.254597 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:10.254661 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:10.257103 (+ 2442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.3:37277: BlockingRecv error: recv got EOF from 127.6.96.3:37277 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 26 11:41:10 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:10 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:10.861325 15697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:10.856281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.1:36399)
0926 11:41:10.856582 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:10.856599 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:10.856693 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:10.856955 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:10.856958 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:10.857124 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:10.857217 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:10.857222 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:10.857989 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:10.857992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:10.858560 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:10.858568 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:10.859023 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:10.859027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:10.859094 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:10.859598 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:10.859610 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:10.861203 (+ 1593us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":90,"threads_started":1}
W20260926 11:41:10.861443 15698 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:10.856366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:36399 (local address 127.6.96.62:41081)
0926 11:41:10.856650 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:10.856654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:10.856667 (+ 13us) server_negotiation.cc:407] Connection header received
0926 11:41:10.856770 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:10.856772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:10.856812 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:10.856884 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:10.857340 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:10.857855 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:10.858650 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:10.858911 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:10.858982 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:10.861332 (+ 2350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.1:36399: BlockingRecv error: recv got EOF from 127.6.96.1:36399 (error 108)
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":83,"threads_started":1}
Sep 26 11:41:11 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:11 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:11.049230 15699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:11.044084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.2:40349)
0926 11:41:11.044324 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:11.044337 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:11.044432 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:11.044631 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:11.044633 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:11.044799 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:11.044892 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:11.044897 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:11.045593 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:11.045596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:11.046158 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:11.046166 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:11.046681 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:11.046684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:11.046755 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:11.047293 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:11.047305 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:11.049071 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":69,"threads_started":1}
W20260926 11:41:11.049309 15698 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:11.044155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:40349 (local address 127.6.96.62:41081)
0926 11:41:11.044276 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:11.044279 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:11.044384 (+ 105us) server_negotiation.cc:407] Connection header received
0926 11:41:11.044495 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:11.044496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:11.044543 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:11.044604 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:11.044971 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:11.045500 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:11.046309 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:11.046531 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:11.046636 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:11.049201 (+ 2565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.2:40349: BlockingRecv error: recv got EOF from 127.6.96.2:40349 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 26 11:41:11 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:11 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.3@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:11.262956 15700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:11.257831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.3:46453)
0926 11:41:11.258092 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:11.258104 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:11.258203 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:11.258396 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:11.258398 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:11.258588 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:11.258678 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:11.258683 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:11.259398 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:11.259400 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:11.260029 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:11.260038 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:11.260530 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:11.260534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:11.260616 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:11.261094 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:11.261105 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:11.262786 (+ 1681us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":86,"threads_started":1}
W20260926 11:41:11.263047 15698 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:11.257936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:46453 (local address 127.6.96.62:41081)
0926 11:41:11.258097 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:11.258100 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:11.258167 (+ 67us) server_negotiation.cc:407] Connection header received
0926 11:41:11.258273 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:11.258275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:11.258321 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:11.258383 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:11.258764 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:11.259316 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:11.260150 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:11.260414 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:11.260523 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:11.262931 (+ 2408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.3:46453: BlockingRecv error: recv got EOF from 127.6.96.3:46453 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 26 11:41:11 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:11 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.1@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:11.867312 15701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:11.862192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.1:36223)
0926 11:41:11.862450 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:11.862464 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:11.862554 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:11.862842 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:11.862845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:11.863012 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:11.863105 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:11.863111 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:11.863840 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:11.863843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:11.864415 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:11.864423 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:11.864823 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:11.864825 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:11.864881 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:11.865331 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:11.865341 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:11.867188 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":68,"threads_started":1}
W20260926 11:41:11.867441 15702 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:11.862291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:36223 (local address 127.6.96.62:41081)
0926 11:41:11.862543 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:11.862546 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:11.862559 (+ 13us) server_negotiation.cc:407] Connection header received
0926 11:41:11.862629 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:11.862631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:11.862677 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:11.862749 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:11.863201 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:11.863737 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:11.864527 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:11.864742 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:11.864813 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:11.867313 (+ 2500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.1:36223: BlockingRecv error: recv got EOF from 127.6.96.1:36223 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":76,"threads_started":1}
Sep 26 11:41:12 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 11:41:12 dist-test-slave-xxxx krb5kdc[14644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.96.2@KRBTEST.COM for oryx/127.6.96.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 11:41:12.055286 15703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:12.050148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:41081 (local address 127.6.96.2:46339)
0926 11:41:12.050386 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:12.050398 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:12.050499 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:12.050732 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:12.050735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:12.050956 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:12.051059 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:12.051065 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:12.051809 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:12.051813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:12.052369 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:12.052377 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:12.052811 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:12.052815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:12.052898 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:12.053389 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:12.053400 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:12.055160 (+ 1760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.96.62:41081: Server oryx/127.6.96.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":74,"threads_started":1}
W20260926 11:41:12.055413 15702 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:12.050210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:46339 (local address 127.6.96.62:41081)
0926 11:41:12.050395 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:12.050399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:12.050472 (+ 73us) server_negotiation.cc:407] Connection header received
0926 11:41:12.050588 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:12.050592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:12.050636 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:12.050696 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:12.051169 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:12.051705 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:12.052481 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:12.052710 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:12.052796 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:12.055296 (+ 2500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.96.2:46339: BlockingRecv error: recv got EOF from 127.6.96.2:46339 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 11:41:12.141062 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 15227
I20260926 11:41:12.145674 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 15379
I20260926 11:41:12.149937 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 15511
I20260926 11:41:12.154348 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 15157
2026-09-26T11:41:12Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17845 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-26T11:41:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:41:12Z Disabled control of system clock
I20260926 11:41:12.181378 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:42389
--webserver_interface=127.6.96.62
--webserver_port=0
--builtin_ntp_servers=127.6.96.20:34435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:42389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 11:41:12.290582 15707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:41:12.290863 15707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:41:12.290897 15707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:41:12.294872 15707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:41:12.294929 15707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:41:12.294942 15707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:41:12.294956 15707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:41:12.294970 15707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:41:12.299741 15707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:34435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:42389
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:42389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:41:12.300781 15707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:41:12.302088 15707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:41:12.310506 15715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:41:12.310508 15712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:41:12.310542 15713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:12.310806 15707 server_base.cc:1034] running on GCE node
I20260926 11:41:12.311522 15707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:41:12.312664 15707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:41:12.313829 15707 hybrid_clock.cc:648] HybridClock initialized: now 1790422872313809 us; error 40 us; skew 500 ppm
I20260926 11:41:12.315797 15707 webserver.cc:460] Webserver started at http://127.6.96.62:46631/ using document root <none> and password file <none>
I20260926 11:41:12.316359 15707 fs_manager.cc:359] Metadata directory not provided
I20260926 11:41:12.316427 15707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:41:12.316643 15707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:41:12.318691 15707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "2ad3f65226e949798c3a819c31a360a7"
format_stamp: "Formatted at 2026-09-26 11:41:12 on dist-test-slave-xxxx"
server_key: "22edbeaede094be0fcc8013977f5e97e"
server_key_iv: "17942e612bcaa699f5865b307de9deb5"
server_key_version: "encryptionkey@0"
I20260926 11:41:12.319342 15707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "2ad3f65226e949798c3a819c31a360a7"
format_stamp: "Formatted at 2026-09-26 11:41:12 on dist-test-slave-xxxx"
server_key: "22edbeaede094be0fcc8013977f5e97e"
server_key_iv: "17942e612bcaa699f5865b307de9deb5"
server_key_version: "encryptionkey@0"
I20260926 11:41:12.323663 15707 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 11:41:12.326067 15721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:12.327382 15707 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 11:41:12.327519 15707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "2ad3f65226e949798c3a819c31a360a7"
format_stamp: "Formatted at 2026-09-26 11:41:12 on dist-test-slave-xxxx"
server_key: "22edbeaede094be0fcc8013977f5e97e"
server_key_iv: "17942e612bcaa699f5865b307de9deb5"
server_key_version: "encryptionkey@0"
I20260926 11:41:12.327626 15707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:41:12.342875 15707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:41:12.343653 15707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:41:12.343853 15707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:41:12.350683 15707 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:42389
I20260926 11:41:12.350703 15773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:42389 every 8 connection(s)
I20260926 11:41:12.351743 15707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:41:12.354990 15774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:41:12.356833 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 15707
I20260926 11:41:12.356945 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:41:12.357241 6528 external_mini_cluster.cc:1428] Setting key 08c79484f42361cad6e22b135ddfc354
I20260926 11:41:12.361726 15774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ad3f65226e949798c3a819c31a360a7: Bootstrap starting.
I20260926 11:41:12.364039 15774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ad3f65226e949798c3a819c31a360a7: Neither blocks nor log segments found. Creating new log.
I20260926 11:41:12.364776 15774 log.cc:824] T 00000000000000000000000000000000 P 2ad3f65226e949798c3a819c31a360a7: Log is configured to *not* fsync() on all Append() calls
I20260926 11:41:12.366413 15777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:12.358709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34294 (local address 127.6.96.62:42389)
0926 11:41:12.359403 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:12.359419 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:12.359456 (+ 37us) server_negotiation.cc:407] Connection header received
0926 11:41:12.360144 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:12.360162 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:12.360598 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:12.360931 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:12.361365 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:12.362340 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:12.363241 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:12.363716 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:12.363862 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:12.364667 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:41:12.364672 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:41:12.364685 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:41:12.365098 (+ 413us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:41:12.365170 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:41:12.365198 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:41:12.365295 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:12.365455 (+ 160us) server_negotiation.cc:299] Negotiation successful
0926 11:41:12.365713 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":142,"threads_started":1}
I20260926 11:41:12.366890 15774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ad3f65226e949798c3a819c31a360a7: No bootstrap required, opened a new log
I20260926 11:41:12.370688 15774 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ad3f65226e949798c3a819c31a360a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad3f65226e949798c3a819c31a360a7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 42389 } }
I20260926 11:41:12.370962 15774 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ad3f65226e949798c3a819c31a360a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:41:12.371024 15774 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ad3f65226e949798c3a819c31a360a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ad3f65226e949798c3a819c31a360a7, State: Initialized, Role: FOLLOWER
I20260926 11:41:12.371450 15774 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ad3f65226e949798c3a819c31a360a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad3f65226e949798c3a819c31a360a7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 42389 } }
I20260926 11:41:12.371585 15774 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ad3f65226e949798c3a819c31a360a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:41:12.371649 15774 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ad3f65226e949798c3a819c31a360a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:41:12.371748 15774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ad3f65226e949798c3a819c31a360a7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:41:12.372677 15774 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ad3f65226e949798c3a819c31a360a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad3f65226e949798c3a819c31a360a7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 42389 } }
I20260926 11:41:12.373028 15774 leader_election.cc:302] T 00000000000000000000000000000000 P 2ad3f65226e949798c3a819c31a360a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ad3f65226e949798c3a819c31a360a7; no voters:
I20260926 11:41:12.373322 15774 leader_election.cc:288] T 00000000000000000000000000000000 P 2ad3f65226e949798c3a819c31a360a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:41:12.373421 15779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ad3f65226e949798c3a819c31a360a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:41:12.373628 15779 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ad3f65226e949798c3a819c31a360a7 [term 1 LEADER]: Becoming Leader. State: Replica: 2ad3f65226e949798c3a819c31a360a7, State: Running, Role: LEADER
I20260926 11:41:12.373979 15779 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ad3f65226e949798c3a819c31a360a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad3f65226e949798c3a819c31a360a7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 42389 } }
I20260926 11:41:12.374584 15774 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ad3f65226e949798c3a819c31a360a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:41:12.375670 15781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ad3f65226e949798c3a819c31a360a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ad3f65226e949798c3a819c31a360a7. Latest consensus state: current_term: 1 leader_uuid: "2ad3f65226e949798c3a819c31a360a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad3f65226e949798c3a819c31a360a7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 42389 } } }
I20260926 11:41:12.375737 15780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ad3f65226e949798c3a819c31a360a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ad3f65226e949798c3a819c31a360a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad3f65226e949798c3a819c31a360a7" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 42389 } } }
I20260926 11:41:12.375932 15780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ad3f65226e949798c3a819c31a360a7 [sys.catalog]: This master's current role is: LEADER
I20260926 11:41:12.375893 15781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ad3f65226e949798c3a819c31a360a7 [sys.catalog]: This master's current role is: LEADER
I20260926 11:41:12.376336 15789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:41:12.379225 15789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:41:12.384367 15789 catalog_manager.cc:1347] Generated new cluster ID: ef19e79d12354f3c9ad1cb91537313d9
I20260926 11:41:12.384443 15789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:41:12.402235 15789 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:41:12.403564 15789 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:41:12.417807 15789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ad3f65226e949798c3a819c31a360a7: Generated new TSK 0
I20260926 11:41:12.418679 15789 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:41:12.428192 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:42389
--builtin_ntp_servers=127.6.96.20:34435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260926 11:41:12.538201 15798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:41:12.538477 15798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:41:12.538512 15798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:41:12.542481 15798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:41:12.542536 15798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:41:12.542586 15798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:41:12.547262 15798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:34435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:42389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:41:12.548288 15798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:41:12.549667 15798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:41:12.559046 15803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:41:12.559144 15804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:41:12.559302 15806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:12.559301 15798 server_base.cc:1034] running on GCE node
I20260926 11:41:12.559742 15798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:41:12.560340 15798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:41:12.561503 15798 hybrid_clock.cc:648] HybridClock initialized: now 1790422872561483 us; error 32 us; skew 500 ppm
I20260926 11:41:12.563362 15798 webserver.cc:460] Webserver started at http://127.6.96.1:38795/ using document root <none> and password file <none>
I20260926 11:41:12.563910 15798 fs_manager.cc:359] Metadata directory not provided
I20260926 11:41:12.563971 15798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:41:12.564160 15798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:41:12.566170 15798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "acf0557445e748a4a2cae05ffe77e1e7"
format_stamp: "Formatted at 2026-09-26 11:41:12 on dist-test-slave-xxxx"
server_key: "f58c778b1e81462debe0a4e58069f0d2"
server_key_iv: "f284eac061c62c4293894cc527f4f7cd"
server_key_version: "encryptionkey@0"
I20260926 11:41:12.566694 15798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "acf0557445e748a4a2cae05ffe77e1e7"
format_stamp: "Formatted at 2026-09-26 11:41:12 on dist-test-slave-xxxx"
server_key: "f58c778b1e81462debe0a4e58069f0d2"
server_key_iv: "f284eac061c62c4293894cc527f4f7cd"
server_key_version: "encryptionkey@0"
I20260926 11:41:12.570588 15798 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260926 11:41:12.573644 15812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:12.575220 15798 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:41:12.575363 15798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "acf0557445e748a4a2cae05ffe77e1e7"
format_stamp: "Formatted at 2026-09-26 11:41:12 on dist-test-slave-xxxx"
server_key: "f58c778b1e81462debe0a4e58069f0d2"
server_key_iv: "f284eac061c62c4293894cc527f4f7cd"
server_key_version: "encryptionkey@0"
I20260926 11:41:12.575501 15798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:41:12.587433 15798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:41:12.588146 15798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:41:12.588323 15798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:41:12.588934 15798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:41:12.589877 15798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:41:12.589924 15798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:12.589977 15798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:41:12.590004 15798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:12.598495 15798 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:33177
I20260926 11:41:12.598520 15925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:33177 every 8 connection(s)
I20260926 11:41:12.599472 15798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:41:12.604053 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 15798
I20260926 11:41:12.604151 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:41:12.604404 6528 external_mini_cluster.cc:1428] Setting key dfa65da134ab6c07c1ca8ecfaa43daf8
I20260926 11:41:12.606354 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:42389
--builtin_ntp_servers=127.6.96.20:34435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 11:41:12.609189 15777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:12.601201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:57357 (local address 127.6.96.62:42389)
0926 11:41:12.601315 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:12.601319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:12.602374 (+ 1055us) server_negotiation.cc:407] Connection header received
0926 11:41:12.603757 (+ 1383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:12.603760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:12.603845 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:12.603918 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:12.604910 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:12.605527 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:12.607171 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:12.607395 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:12.607489 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:12.608674 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:41:12.608678 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:41:12.608681 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:41:12.608694 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:41:12.608715 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:41:12.608724 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:41:12.608795 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:12.608953 (+ 158us) server_negotiation.cc:299] Negotiation successful
0926 11:41:12.609018 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 11:41:12.610049 15928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:12.601594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:42389 (local address 127.6.96.1:57357)
0926 11:41:12.602221 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:12.602263 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:12.603528 (+ 1265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:12.604101 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:12.604111 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:12.604220 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:12.604750 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:12.604769 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:12.605691 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:12.605694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:12.607052 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:12.607064 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:12.607554 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:12.607558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:12.607743 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:12.608394 (+ 651us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 11:41:12.608396 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:12.608441 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 11:41:12.608512 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:41:12.608819 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:41:12.608826 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:41:12.608831 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 11:41:12.608939 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 11:41:12.609182 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":107,"threads_started":1}
I20260926 11:41:12.611289 15926 heartbeater.cc:347] Connected to a master server at 127.6.96.62:42389
I20260926 11:41:12.611538 15926 heartbeater.cc:464] Registering TS with master...
I20260926 11:41:12.612073 15926 heartbeater.cc:511] Master 127.6.96.62:42389 requested a full tablet report, sending...
I20260926 11:41:12.613596 15738 ts_manager.cc:194] Registered new tserver with Master: acf0557445e748a4a2cae05ffe77e1e7 (127.6.96.1:33177)
I20260926 11:41:12.614797 15738 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.96.1:57357
W20260926 11:41:12.714529 15929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:41:12.714803 15929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:41:12.714875 15929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:41:12.719636 15929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:41:12.719745 15929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:41:12.719843 15929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:41:12.726505 15929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:34435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:42389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:41:12.727941 15929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:41:12.729722 15929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:41:12.741415 15935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:41:12.741475 15934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:41:12.741406 15937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:12.742126 15929 server_base.cc:1034] running on GCE node
I20260926 11:41:12.742601 15929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:41:12.743333 15929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:41:12.744498 15929 hybrid_clock.cc:648] HybridClock initialized: now 1790422872744478 us; error 36 us; skew 500 ppm
I20260926 11:41:12.746284 15929 webserver.cc:460] Webserver started at http://127.6.96.2:43655/ using document root <none> and password file <none>
I20260926 11:41:12.746896 15929 fs_manager.cc:359] Metadata directory not provided
I20260926 11:41:12.746969 15929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:41:12.747169 15929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:41:12.749172 15929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "ffa6a920f8104dcfbeb27cd3e9000930"
format_stamp: "Formatted at 2026-09-26 11:41:12 on dist-test-slave-xxxx"
server_key: "eda6057ca3ae9d3c1d52f35010d19a25"
server_key_iv: "37683285ac8cb94b3d9d48f582ca4a45"
server_key_version: "encryptionkey@0"
I20260926 11:41:12.749716 15929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "ffa6a920f8104dcfbeb27cd3e9000930"
format_stamp: "Formatted at 2026-09-26 11:41:12 on dist-test-slave-xxxx"
server_key: "eda6057ca3ae9d3c1d52f35010d19a25"
server_key_iv: "37683285ac8cb94b3d9d48f582ca4a45"
server_key_version: "encryptionkey@0"
I20260926 11:41:12.753502 15929 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 11:41:12.755911 15943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:12.757045 15929 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 11:41:12.757140 15929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "ffa6a920f8104dcfbeb27cd3e9000930"
format_stamp: "Formatted at 2026-09-26 11:41:12 on dist-test-slave-xxxx"
server_key: "eda6057ca3ae9d3c1d52f35010d19a25"
server_key_iv: "37683285ac8cb94b3d9d48f582ca4a45"
server_key_version: "encryptionkey@0"
I20260926 11:41:12.757230 15929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:41:12.783946 15929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:41:12.784839 15929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:41:12.785035 15929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:41:12.785617 15929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:41:12.786559 15929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:41:12.786607 15929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:12.786659 15929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:41:12.786686 15929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:12.794971 15929 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:42063
I20260926 11:41:12.795006 16056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:42063 every 8 connection(s)
I20260926 11:41:12.795856 15929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:41:12.802839 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 15929
I20260926 11:41:12.802958 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:41:12.803241 6528 external_mini_cluster.cc:1428] Setting key c78c2f568984b7163778d97a3afbb00f
I20260926 11:41:12.804991 15777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:12.797693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:56969 (local address 127.6.96.62:42389)
0926 11:41:12.797848 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:12.797852 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:12.798879 (+ 1027us) server_negotiation.cc:407] Connection header received
0926 11:41:12.800352 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:12.800355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:12.800400 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:12.800470 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:12.801344 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:12.801852 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:12.803095 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:12.803308 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:12.803398 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:12.804496 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:41:12.804499 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:41:12.804502 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:41:12.804515 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:41:12.804537 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:41:12.804548 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:41:12.804604 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:12.804792 (+ 188us) server_negotiation.cc:299] Negotiation successful
0926 11:41:12.804856 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 11:41:12.805274 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:42389
--builtin_ntp_servers=127.6.96.20:34435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 11:41:12.805856 16059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:12.798082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:42389 (local address 127.6.96.2:56969)
0926 11:41:12.798716 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:12.798759 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:12.800078 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:12.800633 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:12.800640 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:12.800727 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:12.801236 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:12.801248 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:12.801938 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:12.801940 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:12.802989 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:12.802999 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:12.803457 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:12.803462 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:12.803649 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:12.804198 (+ 549us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 11:41:12.804200 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:12.804253 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 11:41:12.804333 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:41:12.804636 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:41:12.804646 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:41:12.804651 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 11:41:12.804776 (+ 125us) client_negotiation.cc:240] Negotiation successful
0926 11:41:12.805001 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":106,"threads_started":1}
I20260926 11:41:12.807188 16057 heartbeater.cc:347] Connected to a master server at 127.6.96.62:42389
I20260926 11:41:12.807438 16057 heartbeater.cc:464] Registering TS with master...
I20260926 11:41:12.807932 16057 heartbeater.cc:511] Master 127.6.96.62:42389 requested a full tablet report, sending...
I20260926 11:41:12.808866 15738 ts_manager.cc:194] Registered new tserver with Master: ffa6a920f8104dcfbeb27cd3e9000930 (127.6.96.2:42063)
I20260926 11:41:12.809410 15738 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.96.2:56969
W20260926 11:41:12.913612 16060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:41:12.913851 16060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:41:12.913900 16060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:41:12.917878 16060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:41:12.917941 16060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:41:12.918011 16060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:41:12.922652 16060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:34435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:42389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:41:12.923652 16060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:41:12.924925 16060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:41:12.934294 16066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:41:12.934290 16065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:12.934384 16060 server_base.cc:1034] running on GCE node
W20260926 11:41:12.934285 16068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:12.935009 16060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:41:12.935621 16060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:41:12.936801 16060 hybrid_clock.cc:648] HybridClock initialized: now 1790422872936782 us; error 32 us; skew 500 ppm
I20260926 11:41:12.938560 16060 webserver.cc:460] Webserver started at http://127.6.96.3:45461/ using document root <none> and password file <none>
I20260926 11:41:12.939148 16060 fs_manager.cc:359] Metadata directory not provided
I20260926 11:41:12.939210 16060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:41:12.939395 16060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:41:12.941381 16060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "652291006c7448b08eeecab9ed91ae7e"
format_stamp: "Formatted at 2026-09-26 11:41:12 on dist-test-slave-xxxx"
server_key: "f63e520d9e8c31f829e6cec246580de2"
server_key_iv: "0700da74914cf3bbc881660d6ff28139"
server_key_version: "encryptionkey@0"
I20260926 11:41:12.941893 16060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "652291006c7448b08eeecab9ed91ae7e"
format_stamp: "Formatted at 2026-09-26 11:41:12 on dist-test-slave-xxxx"
server_key: "f63e520d9e8c31f829e6cec246580de2"
server_key_iv: "0700da74914cf3bbc881660d6ff28139"
server_key_version: "encryptionkey@0"
I20260926 11:41:12.945386 16060 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260926 11:41:12.947836 16074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:12.949033 16060 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 11:41:12.949132 16060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "652291006c7448b08eeecab9ed91ae7e"
format_stamp: "Formatted at 2026-09-26 11:41:12 on dist-test-slave-xxxx"
server_key: "f63e520d9e8c31f829e6cec246580de2"
server_key_iv: "0700da74914cf3bbc881660d6ff28139"
server_key_version: "encryptionkey@0"
I20260926 11:41:12.949227 16060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:41:12.970607 16060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:41:12.971551 16060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:41:12.971750 16060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:41:12.972450 16060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:41:12.973676 16060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:41:12.973737 16060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:12.973791 16060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:41:12.973822 16060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:12.982184 16060 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:38421
I20260926 11:41:12.982228 16187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:38421 every 8 connection(s)
I20260926 11:41:12.983112 16060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:41:12.991284 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 16060
I20260926 11:41:12.991390 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:41:12.991638 6528 external_mini_cluster.cc:1428] Setting key dc147827b4a61bd203cce4e86c7227c8
I20260926 11:41:12.992228 15777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:12.984825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:39325 (local address 127.6.96.62:42389)
0926 11:41:12.984957 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:12.984961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:12.986041 (+ 1080us) server_negotiation.cc:407] Connection header received
0926 11:41:12.987474 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:12.987477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:12.987533 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:12.987621 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:12.988584 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:12.989125 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:12.990262 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:12.990486 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:12.990581 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:12.991765 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:41:12.991769 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:41:12.991772 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:41:12.991786 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:41:12.991810 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:41:12.991820 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:41:12.991891 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:12.992031 (+ 140us) server_negotiation.cc:299] Negotiation successful
0926 11:41:12.992091 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 11:41:12.993309 16190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:12.985230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:42389 (local address 127.6.96.3:39325)
0926 11:41:12.985888 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:12.985929 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:12.987266 (+ 1337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:12.987811 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:12.987818 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:12.987907 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:12.988425 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:12.988438 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:12.989250 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:12.989253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:12.990144 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:12.990154 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:12.990633 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:12.990638 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:12.990795 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:12.991453 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 11:41:12.991455 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:12.991512 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 11:41:12.991581 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:41:12.991896 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:41:12.991902 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:41:12.991907 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 11:41:12.992042 (+ 135us) client_negotiation.cc:240] Negotiation successful
0926 11:41:12.992326 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":110,"threads_started":1}
I20260926 11:41:12.994464 16188 heartbeater.cc:347] Connected to a master server at 127.6.96.62:42389
I20260926 11:41:12.994705 16188 heartbeater.cc:464] Registering TS with master...
I20260926 11:41:12.995272 16188 heartbeater.cc:511] Master 127.6.96.62:42389 requested a full tablet report, sending...
I20260926 11:41:12.996228 15738 ts_manager.cc:194] Registered new tserver with Master: 652291006c7448b08eeecab9ed91ae7e (127.6.96.3:38421)
I20260926 11:41:12.996763 15738 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.96.3:39325
I20260926 11:41:13.005398 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 11:41:13.008111 15777 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:13.007202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34300 (local address 127.6.96.62:42389)
0926 11:41:13.007301 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:13.007305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:13.007512 (+ 207us) server_negotiation.cc:407] Connection header received
0926 11:41:13.007643 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:13.007646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:13.007690 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:13.007751 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:13.007998 (+ 247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34300: BlockingRecv error: recv got EOF from 127.0.0.1:34300 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 11:41:13.008369 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 15798
I20260926 11:41:13.012951 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 15929
I20260926 11:41:13.017369 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 16060
I20260926 11:41:13.021632 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 15707
2026-09-26T11:41:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (866 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-26T11:41:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:41:13Z Disabled control of system clock
I20260926 11:41:13.047912 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:43909
--webserver_interface=127.6.96.62
--webserver_port=0
--builtin_ntp_servers=127.6.96.20:36381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:43909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 11:41:13.154815 16199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:41:13.155118 16199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:41:13.155155 16199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:41:13.159212 16199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:41:13.159277 16199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:41:13.159292 16199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:41:13.159308 16199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:41:13.159322 16199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:41:13.164186 16199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:36381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:43909
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:43909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:41:13.165278 16199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:41:13.166677 16199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:41:13.174983 16204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:41:13.175009 16205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:13.175091 16199 server_base.cc:1034] running on GCE node
W20260926 11:41:13.175001 16207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:13.175850 16199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:41:13.176983 16199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:41:13.178187 16199 hybrid_clock.cc:648] HybridClock initialized: now 1790422873178159 us; error 42 us; skew 500 ppm
I20260926 11:41:13.180204 16199 webserver.cc:460] Webserver started at http://127.6.96.62:36879/ using document root <none> and password file <none>
I20260926 11:41:13.180786 16199 fs_manager.cc:359] Metadata directory not provided
I20260926 11:41:13.180852 16199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:41:13.181052 16199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:41:13.183323 16199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "a7bea61a3fc643999515e492a1784944"
format_stamp: "Formatted at 2026-09-26 11:41:13 on dist-test-slave-xxxx"
server_key: "bad833afdeaf82cd6b303ce72a88ca37"
server_key_iv: "94ac1efd06bcaa8be7ced33c3e838025"
server_key_version: "encryptionkey@0"
I20260926 11:41:13.183894 16199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "a7bea61a3fc643999515e492a1784944"
format_stamp: "Formatted at 2026-09-26 11:41:13 on dist-test-slave-xxxx"
server_key: "bad833afdeaf82cd6b303ce72a88ca37"
server_key_iv: "94ac1efd06bcaa8be7ced33c3e838025"
server_key_version: "encryptionkey@0"
I20260926 11:41:13.187669 16199 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 11:41:13.190407 16213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:13.191823 16199 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:41:13.191954 16199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "a7bea61a3fc643999515e492a1784944"
format_stamp: "Formatted at 2026-09-26 11:41:13 on dist-test-slave-xxxx"
server_key: "bad833afdeaf82cd6b303ce72a88ca37"
server_key_iv: "94ac1efd06bcaa8be7ced33c3e838025"
server_key_version: "encryptionkey@0"
I20260926 11:41:13.192072 16199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:41:13.196630 16199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:41:13.197237 16199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:41:13.197405 16199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:41:13.204291 16199 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:43909
I20260926 11:41:13.204310 16265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:43909 every 8 connection(s)
I20260926 11:41:13.205235 16199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:41:13.208554 16266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:41:13.213207 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 16199
I20260926 11:41:13.213323 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:41:13.213618 6528 external_mini_cluster.cc:1428] Setting key 90f21985f485a8e7411a16cd00a2e01d
I20260926 11:41:13.214447 16266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7bea61a3fc643999515e492a1784944: Bootstrap starting.
I20260926 11:41:13.216728 16266 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7bea61a3fc643999515e492a1784944: Neither blocks nor log segments found. Creating new log.
I20260926 11:41:13.217561 16266 log.cc:824] T 00000000000000000000000000000000 P a7bea61a3fc643999515e492a1784944: Log is configured to *not* fsync() on all Append() calls
I20260926 11:41:13.219058 16269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:13.215054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36308 (local address 127.6.96.62:43909)
0926 11:41:13.215697 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:13.215711 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:13.215746 (+ 35us) server_negotiation.cc:407] Connection header received
0926 11:41:13.216359 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:13.216375 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:13.216731 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:13.217038 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:13.217784 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:41:13.217788 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:41:13.217802 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:41:13.217812 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:41:13.217869 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:41:13.217893 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:41:13.217987 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:13.218115 (+ 128us) server_negotiation.cc:299] Negotiation successful
0926 11:41:13.218340 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":106,"threads_started":1}
I20260926 11:41:13.219547 16266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7bea61a3fc643999515e492a1784944: No bootstrap required, opened a new log
I20260926 11:41:13.223281 16266 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7bea61a3fc643999515e492a1784944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7bea61a3fc643999515e492a1784944" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43909 } }
I20260926 11:41:13.223538 16266 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7bea61a3fc643999515e492a1784944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:41:13.223598 16266 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7bea61a3fc643999515e492a1784944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7bea61a3fc643999515e492a1784944, State: Initialized, Role: FOLLOWER
I20260926 11:41:13.224061 16266 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7bea61a3fc643999515e492a1784944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7bea61a3fc643999515e492a1784944" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43909 } }
I20260926 11:41:13.224222 16266 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7bea61a3fc643999515e492a1784944 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:41:13.224290 16266 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7bea61a3fc643999515e492a1784944 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:41:13.224390 16266 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7bea61a3fc643999515e492a1784944 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:41:13.225244 16266 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7bea61a3fc643999515e492a1784944 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7bea61a3fc643999515e492a1784944" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43909 } }
I20260926 11:41:13.225549 16266 leader_election.cc:302] T 00000000000000000000000000000000 P a7bea61a3fc643999515e492a1784944 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7bea61a3fc643999515e492a1784944; no voters:
I20260926 11:41:13.225816 16266 leader_election.cc:288] T 00000000000000000000000000000000 P a7bea61a3fc643999515e492a1784944 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:41:13.225903 16271 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7bea61a3fc643999515e492a1784944 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:41:13.226122 16271 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7bea61a3fc643999515e492a1784944 [term 1 LEADER]: Becoming Leader. State: Replica: a7bea61a3fc643999515e492a1784944, State: Running, Role: LEADER
I20260926 11:41:13.226424 16271 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7bea61a3fc643999515e492a1784944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7bea61a3fc643999515e492a1784944" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43909 } }
I20260926 11:41:13.227007 16266 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7bea61a3fc643999515e492a1784944 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:41:13.228135 16273 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7bea61a3fc643999515e492a1784944 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7bea61a3fc643999515e492a1784944. Latest consensus state: current_term: 1 leader_uuid: "a7bea61a3fc643999515e492a1784944" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7bea61a3fc643999515e492a1784944" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43909 } } }
I20260926 11:41:13.228157 16272 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7bea61a3fc643999515e492a1784944 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7bea61a3fc643999515e492a1784944" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7bea61a3fc643999515e492a1784944" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43909 } } }
I20260926 11:41:13.228250 16273 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7bea61a3fc643999515e492a1784944 [sys.catalog]: This master's current role is: LEADER
I20260926 11:41:13.228257 16272 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7bea61a3fc643999515e492a1784944 [sys.catalog]: This master's current role is: LEADER
I20260926 11:41:13.228574 16280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:41:13.231221 16280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:41:13.236541 16280 catalog_manager.cc:1347] Generated new cluster ID: 37db07429971449db592602839e79e86
I20260926 11:41:13.236601 16280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:41:13.248219 16280 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:41:13.248607 16280 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:41:13.256456 16280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7bea61a3fc643999515e492a1784944: Generated new TSK 0
I20260926 11:41:13.257326 16280 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:41:13.260286 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:43909
--builtin_ntp_servers=127.6.96.20:36381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 11:41:13.368904 16290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:41:13.369139 16290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:41:13.369177 16290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:41:13.373087 16290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:41:13.373138 16290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:41:13.373193 16290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:41:13.377919 16290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:36381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:43909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:41:13.378934 16290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:41:13.380214 16290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:41:13.389371 16295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:41:13.389447 16296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:13.389430 16290 server_base.cc:1034] running on GCE node
W20260926 11:41:13.389371 16298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:13.389961 16290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:41:13.390576 16290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:41:13.391726 16290 hybrid_clock.cc:648] HybridClock initialized: now 1790422873391695 us; error 43 us; skew 500 ppm
I20260926 11:41:13.401257 16290 webserver.cc:460] Webserver started at http://127.6.96.1:33275/ using document root <none> and password file <none>
I20260926 11:41:13.401870 16290 fs_manager.cc:359] Metadata directory not provided
I20260926 11:41:13.401940 16290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:41:13.402161 16290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:41:13.404279 16290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "dd7308269db747c58ae036a351542ec4"
format_stamp: "Formatted at 2026-09-26 11:41:13 on dist-test-slave-xxxx"
server_key: "f89be0e8ef0ccd1330d1a02ce4f86d2f"
server_key_iv: "e5bd04ab3222d60ade1919a48c38c0e4"
server_key_version: "encryptionkey@0"
I20260926 11:41:13.404809 16290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "dd7308269db747c58ae036a351542ec4"
format_stamp: "Formatted at 2026-09-26 11:41:13 on dist-test-slave-xxxx"
server_key: "f89be0e8ef0ccd1330d1a02ce4f86d2f"
server_key_iv: "e5bd04ab3222d60ade1919a48c38c0e4"
server_key_version: "encryptionkey@0"
I20260926 11:41:13.408691 16290 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 11:41:13.411188 16304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:13.412427 16290 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:41:13.412621 16290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "dd7308269db747c58ae036a351542ec4"
format_stamp: "Formatted at 2026-09-26 11:41:13 on dist-test-slave-xxxx"
server_key: "f89be0e8ef0ccd1330d1a02ce4f86d2f"
server_key_iv: "e5bd04ab3222d60ade1919a48c38c0e4"
server_key_version: "encryptionkey@0"
I20260926 11:41:13.412771 16290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:41:13.417398 16290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:41:13.417984 16290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:41:13.418133 16290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:41:13.418654 16290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:41:13.419606 16290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:41:13.419652 16290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:13.419703 16290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:41:13.419727 16290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:13.427812 16290 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:34715
I20260926 11:41:13.427881 16417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:34715 every 8 connection(s)
I20260926 11:41:13.428728 16290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:41:13.434500 16269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:13.430498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:54171 (local address 127.6.96.62:43909)
0926 11:41:13.430658 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:13.430662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:13.431752 (+ 1090us) server_negotiation.cc:407] Connection header received
0926 11:41:13.433001 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:13.433003 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:13.433042 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:13.433122 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:13.434152 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:41:13.434155 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:41:13.434157 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:41:13.434168 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:41:13.434190 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:41:13.434198 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:41:13.434278 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:13.434364 (+ 86us) server_negotiation.cc:299] Negotiation successful
0926 11:41:13.434399 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 11:41:13.435393 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:13.430891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:43909 (local address 127.6.96.1:54171)
0926 11:41:13.431612 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:13.431655 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:13.432804 (+ 1149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:13.433269 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:13.433276 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:13.433366 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:13.433890 (+ 524us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 11:41:13.433892 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:13.433933 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 11:41:13.434000 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:41:13.434264 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:41:13.434269 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:41:13.434272 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 11:41:13.434394 (+ 122us) client_negotiation.cc:240] Negotiation successful
0926 11:41:13.434608 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":105,"threads_started":1}
I20260926 11:41:13.436272 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 16290
I20260926 11:41:13.436370 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:41:13.436559 16418 heartbeater.cc:347] Connected to a master server at 127.6.96.62:43909
I20260926 11:41:13.436677 6528 external_mini_cluster.cc:1428] Setting key d2b1cac2c526e7391afb8a06ced24705
I20260926 11:41:13.436794 16418 heartbeater.cc:464] Registering TS with master...
I20260926 11:41:13.437219 16418 heartbeater.cc:511] Master 127.6.96.62:43909 requested a full tablet report, sending...
I20260926 11:41:13.438439 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:43909
--builtin_ntp_servers=127.6.96.20:36381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260926 11:41:13.438722 16230 ts_manager.cc:194] Registered new tserver with Master: dd7308269db747c58ae036a351542ec4 (127.6.96.1:34715)
W20260926 11:41:13.545825 16421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:41:13.546079 16421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:41:13.546134 16421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:41:13.550457 16421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:41:13.550515 16421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:41:13.550590 16421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:41:13.555320 16421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:36381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:43909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:41:13.556325 16421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:41:13.557644 16421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:41:13.566751 16427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:41:13.566767 16426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:13.566880 16421 server_base.cc:1034] running on GCE node
W20260926 11:41:13.566761 16429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:13.567409 16421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:41:13.568141 16421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:41:13.569321 16421 hybrid_clock.cc:648] HybridClock initialized: now 1790422873569267 us; error 63 us; skew 500 ppm
I20260926 11:41:13.571249 16421 webserver.cc:460] Webserver started at http://127.6.96.2:42949/ using document root <none> and password file <none>
I20260926 11:41:13.571911 16421 fs_manager.cc:359] Metadata directory not provided
I20260926 11:41:13.571975 16421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:41:13.572162 16421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:41:13.574327 16421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "c291dd9bd5894034891e2c9951499c66"
format_stamp: "Formatted at 2026-09-26 11:41:13 on dist-test-slave-xxxx"
server_key: "f1269ae65f273522b1f633a25684c49b"
server_key_iv: "f2f68df87a89715b07f7928492ddcf6f"
server_key_version: "encryptionkey@0"
I20260926 11:41:13.574920 16421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "c291dd9bd5894034891e2c9951499c66"
format_stamp: "Formatted at 2026-09-26 11:41:13 on dist-test-slave-xxxx"
server_key: "f1269ae65f273522b1f633a25684c49b"
server_key_iv: "f2f68df87a89715b07f7928492ddcf6f"
server_key_version: "encryptionkey@0"
I20260926 11:41:13.578572 16421 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:41:13.581140 16435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:13.582396 16421 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:41:13.582507 16421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "c291dd9bd5894034891e2c9951499c66"
format_stamp: "Formatted at 2026-09-26 11:41:13 on dist-test-slave-xxxx"
server_key: "f1269ae65f273522b1f633a25684c49b"
server_key_iv: "f2f68df87a89715b07f7928492ddcf6f"
server_key_version: "encryptionkey@0"
I20260926 11:41:13.582644 16421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:41:13.587419 16421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:41:13.587994 16421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:41:13.588142 16421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:41:13.588666 16421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:41:13.589542 16421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:41:13.589596 16421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:13.589633 16421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:41:13.589649 16421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:13.598019 16421 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:46235
I20260926 11:41:13.598063 16548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:46235 every 8 connection(s)
I20260926 11:41:13.598978 16421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:41:13.604676 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 16421
I20260926 11:41:13.604801 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:41:13.604794 16269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:13.600697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:32995 (local address 127.6.96.62:43909)
0926 11:41:13.600814 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:13.600817 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:13.601920 (+ 1103us) server_negotiation.cc:407] Connection header received
0926 11:41:13.603180 (+ 1260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:13.603183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:13.603220 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:13.603292 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:13.604322 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:41:13.604325 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:41:13.604327 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:41:13.604337 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:41:13.604355 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:41:13.604363 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:41:13.604462 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:13.604611 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 11:41:13.604658 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 11:41:13.605099 6528 external_mini_cluster.cc:1428] Setting key db0cb0cc750d1f089bdc19887caeeeb1
I20260926 11:41:13.605620 16551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:13.601133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:43909 (local address 127.6.96.2:32995)
0926 11:41:13.601775 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:13.601814 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:13.603000 (+ 1186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:13.603434 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:13.603440 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:13.603552 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:13.604076 (+ 524us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 11:41:13.604078 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:13.604118 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 11:41:13.604187 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:41:13.604460 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:41:13.604468 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:41:13.604473 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 11:41:13.604627 (+ 154us) client_negotiation.cc:240] Negotiation successful
0926 11:41:13.604833 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":110,"threads_started":1}
I20260926 11:41:13.606709 16549 heartbeater.cc:347] Connected to a master server at 127.6.96.62:43909
I20260926 11:41:13.606968 16549 heartbeater.cc:464] Registering TS with master...
I20260926 11:41:13.607028 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:43909
--builtin_ntp_servers=127.6.96.20:36381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260926 11:41:13.607402 16549 heartbeater.cc:511] Master 127.6.96.62:43909 requested a full tablet report, sending...
I20260926 11:41:13.608536 16229 ts_manager.cc:194] Registered new tserver with Master: c291dd9bd5894034891e2c9951499c66 (127.6.96.2:46235)
W20260926 11:41:13.715391 16552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:41:13.715636 16552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:41:13.715672 16552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:41:13.719738 16552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:41:13.719794 16552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:41:13.719851 16552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:41:13.724469 16552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:36381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:43909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:41:13.725441 16552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:41:13.726743 16552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:41:13.735992 16560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:41:13.736011 16557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:13.736160 16552 server_base.cc:1034] running on GCE node
W20260926 11:41:13.736011 16558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:13.736701 16552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:41:13.737313 16552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:41:13.738469 16552 hybrid_clock.cc:648] HybridClock initialized: now 1790422873738453 us; error 31 us; skew 500 ppm
I20260926 11:41:13.740263 16552 webserver.cc:460] Webserver started at http://127.6.96.3:38009/ using document root <none> and password file <none>
I20260926 11:41:13.740808 16552 fs_manager.cc:359] Metadata directory not provided
I20260926 11:41:13.740871 16552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:41:13.741057 16552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:41:13.743142 16552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "fa1e589a6e0541f59b2b48fedcae2b51"
format_stamp: "Formatted at 2026-09-26 11:41:13 on dist-test-slave-xxxx"
server_key: "25139c787123be1ac5b49b1325183256"
server_key_iv: "8716876445a3cbd9c360c192d452d6ab"
server_key_version: "encryptionkey@0"
I20260926 11:41:13.743664 16552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "fa1e589a6e0541f59b2b48fedcae2b51"
format_stamp: "Formatted at 2026-09-26 11:41:13 on dist-test-slave-xxxx"
server_key: "25139c787123be1ac5b49b1325183256"
server_key_iv: "8716876445a3cbd9c360c192d452d6ab"
server_key_version: "encryptionkey@0"
I20260926 11:41:13.747109 16552 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 11:41:13.749498 16566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:13.750761 16552 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:41:13.750882 16552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "fa1e589a6e0541f59b2b48fedcae2b51"
format_stamp: "Formatted at 2026-09-26 11:41:13 on dist-test-slave-xxxx"
server_key: "25139c787123be1ac5b49b1325183256"
server_key_iv: "8716876445a3cbd9c360c192d452d6ab"
server_key_version: "encryptionkey@0"
I20260926 11:41:13.750986 16552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:41:13.755475 16552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:41:13.756080 16552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:41:13.756245 16552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:41:13.756783 16552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:41:13.757665 16552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:41:13.757711 16552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:13.757759 16552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:41:13.757786 16552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:13.765854 16552 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:33123
I20260926 11:41:13.765890 16679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:33123 every 8 connection(s)
I20260926 11:41:13.766747 16552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:41:13.772379 16269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:13.768459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:60129 (local address 127.6.96.62:43909)
0926 11:41:13.768620 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:13.768624 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:13.769665 (+ 1041us) server_negotiation.cc:407] Connection header received
0926 11:41:13.770948 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:13.770951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:13.770988 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:13.771057 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:13.772036 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:41:13.772039 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:41:13.772041 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:41:13.772051 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:41:13.772072 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:41:13.772081 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:41:13.772155 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:13.772243 (+ 88us) server_negotiation.cc:299] Negotiation successful
0926 11:41:13.772278 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 11:41:13.772964 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 16552
I20260926 11:41:13.773074 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:41:13.773361 6528 external_mini_cluster.cc:1428] Setting key 0f39b6525b099430ef9eb1390f32187c
I20260926 11:41:13.773375 16682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:13.768882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:43909 (local address 127.6.96.3:60129)
0926 11:41:13.769518 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:13.769567 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:13.770711 (+ 1144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:13.771193 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:13.771199 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:13.771286 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:13.771804 (+ 518us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 11:41:13.771806 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:13.771845 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 11:41:13.771913 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:41:13.772147 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:41:13.772151 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:41:13.772155 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 11:41:13.772275 (+ 120us) client_negotiation.cc:240] Negotiation successful
0926 11:41:13.772466 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":114,"threads_started":1}
I20260926 11:41:13.774454 16680 heartbeater.cc:347] Connected to a master server at 127.6.96.62:43909
I20260926 11:41:13.774701 16680 heartbeater.cc:464] Registering TS with master...
I20260926 11:41:13.775154 16680 heartbeater.cc:511] Master 127.6.96.62:43909 requested a full tablet report, sending...
I20260926 11:41:13.776033 16230 ts_manager.cc:194] Registered new tserver with Master: fa1e589a6e0541f59b2b48fedcae2b51 (127.6.96.3:33123)
I20260926 11:41:13.787122 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 11:41:13.789713 16269 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:13.788932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36310 (local address 127.6.96.62:43909)
0926 11:41:13.789071 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:13.789075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:13.789182 (+ 107us) server_negotiation.cc:407] Connection header received
0926 11:41:13.789285 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:13.789287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:13.789320 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:13.789374 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:13.789593 (+ 219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36310: BlockingRecv error: recv got EOF from 127.0.0.1:36310 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 11:41:13.789937 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 16290
I20260926 11:41:13.795274 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 16421
I20260926 11:41:13.799466 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 16552
I20260926 11:41:13.804039 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 16199
2026-09-26T11:41:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (782 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:41:13 dist-test-slave-xxxx krb5kdc[16690](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:41:13 dist-test-slave-xxxx krb5kdc[16690](info): set up 2 sockets
Sep 26 11:41:13 dist-test-slave-xxxx krb5kdc[16690](info): commencing operation
krb5kdc: starting...
W20260926 11:41:14.755504 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:41:14 dist-test-slave-xxxx krb5kdc[16690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:41:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:41:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:41:14.871318 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:40865
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:41771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:40865
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:41:14.976927 16703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:41:14.977216 16703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:41:14.977275 16703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:41:14.981334 16703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:41:14.981397 16703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:41:14.981423 16703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:41:14.981444 16703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:41:14.981472 16703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:41:14.986800 16703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:41771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:40865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:40865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:41:14.987877 16703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:41:14.988871 16703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:41:14.996963 16708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:41:14.996989 16709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:41:14.997007 16711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:14.997109 16703 server_base.cc:1034] running on GCE node
I20260926 11:41:14.997716 16703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:41:14.998739 16703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:41:14.999886 16703 hybrid_clock.cc:648] HybridClock initialized: now 1790422874999858 us; error 40 us; skew 500 ppm
Sep 26 11:41:14 dist-test-slave-xxxx krb5kdc[16690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422874, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:41:15.002463 16703 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:41:15.003355 16703 webserver.cc:460] Webserver started at http://127.6.96.62:40829/ using document root <none> and password file <none>
I20260926 11:41:15.003875 16703 fs_manager.cc:359] Metadata directory not provided
I20260926 11:41:15.003932 16703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:41:15.004137 16703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:41:15.006071 16703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "bbb5238b828b442c887c28a932e8511e"
format_stamp: "Formatted at 2026-09-26 11:41:15 on dist-test-slave-xxxx"
server_key: "687f1d346111f9a44e07dddc2b3659ce"
server_key_iv: "f3f996f87f3cdfdaccf2670a77e3d96b"
server_key_version: "encryptionkey@0"
I20260926 11:41:15.006588 16703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "bbb5238b828b442c887c28a932e8511e"
format_stamp: "Formatted at 2026-09-26 11:41:15 on dist-test-slave-xxxx"
server_key: "687f1d346111f9a44e07dddc2b3659ce"
server_key_iv: "f3f996f87f3cdfdaccf2670a77e3d96b"
server_key_version: "encryptionkey@0"
I20260926 11:41:15.010018 16703 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 11:41:15.012379 16718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:15.013571 16703 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 11:41:15.013670 16703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "bbb5238b828b442c887c28a932e8511e"
format_stamp: "Formatted at 2026-09-26 11:41:15 on dist-test-slave-xxxx"
server_key: "687f1d346111f9a44e07dddc2b3659ce"
server_key_iv: "f3f996f87f3cdfdaccf2670a77e3d96b"
server_key_version: "encryptionkey@0"
I20260926 11:41:15.013761 16703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:41:15.026614 16703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:41:15.029531 16703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:41:15.029693 16703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:41:15.036249 16703 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:40865
I20260926 11:41:15.036495 16770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:40865 every 8 connection(s)
I20260926 11:41:15.037292 16703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:41:15.039983 16771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:41:15.045326 16771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbb5238b828b442c887c28a932e8511e: Bootstrap starting.
I20260926 11:41:15.047024 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 16703
I20260926 11:41:15.047147 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:41:15.047418 6528 external_mini_cluster.cc:1428] Setting key 4255371e4b3bd38e642df7f6011c73e4
I20260926 11:41:15.047976 16771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbb5238b828b442c887c28a932e8511e: Neither blocks nor log segments found. Creating new log.
I20260926 11:41:15.048801 16771 log.cc:824] T 00000000000000000000000000000000 P bbb5238b828b442c887c28a932e8511e: Log is configured to *not* fsync() on all Append() calls
I20260926 11:41:15.050736 16771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbb5238b828b442c887c28a932e8511e: No bootstrap required, opened a new log
I20260926 11:41:15.054651 16771 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbb5238b828b442c887c28a932e8511e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb5238b828b442c887c28a932e8511e" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40865 } }
I20260926 11:41:15.054948 16771 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbb5238b828b442c887c28a932e8511e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:41:15.055008 16771 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbb5238b828b442c887c28a932e8511e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbb5238b828b442c887c28a932e8511e, State: Initialized, Role: FOLLOWER
I20260926 11:41:15.055518 16771 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbb5238b828b442c887c28a932e8511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb5238b828b442c887c28a932e8511e" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40865 } }
I20260926 11:41:15.055665 16771 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbb5238b828b442c887c28a932e8511e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:41:15.055728 16771 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbb5238b828b442c887c28a932e8511e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:41:15.055820 16771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbb5238b828b442c887c28a932e8511e [term 0 FOLLOWER]: Advancing to term 1
Sep 26 11:41:15 dist-test-slave-xxxx krb5kdc[16690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:41:15.056789 16771 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbb5238b828b442c887c28a932e8511e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb5238b828b442c887c28a932e8511e" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40865 } }
I20260926 11:41:15.057150 16771 leader_election.cc:302] T 00000000000000000000000000000000 P bbb5238b828b442c887c28a932e8511e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbb5238b828b442c887c28a932e8511e; no voters:
I20260926 11:41:15.057504 16771 leader_election.cc:288] T 00000000000000000000000000000000 P bbb5238b828b442c887c28a932e8511e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:41:15.057718 16776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbb5238b828b442c887c28a932e8511e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:41:15.057921 16776 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbb5238b828b442c887c28a932e8511e [term 1 LEADER]: Becoming Leader. State: Replica: bbb5238b828b442c887c28a932e8511e, State: Running, Role: LEADER
I20260926 11:41:15.058197 16776 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbb5238b828b442c887c28a932e8511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb5238b828b442c887c28a932e8511e" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40865 } }
I20260926 11:41:15.058813 16771 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbb5238b828b442c887c28a932e8511e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:41:15.059360 16778 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbb5238b828b442c887c28a932e8511e [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbb5238b828b442c887c28a932e8511e. Latest consensus state: current_term: 1 leader_uuid: "bbb5238b828b442c887c28a932e8511e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb5238b828b442c887c28a932e8511e" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40865 } } }
I20260926 11:41:15.059471 16778 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbb5238b828b442c887c28a932e8511e [sys.catalog]: This master's current role is: LEADER
I20260926 11:41:15.060050 16783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:41:15.060006 16777 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbb5238b828b442c887c28a932e8511e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbb5238b828b442c887c28a932e8511e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb5238b828b442c887c28a932e8511e" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40865 } } }
I20260926 11:41:15.060122 16777 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbb5238b828b442c887c28a932e8511e [sys.catalog]: This master's current role is: LEADER
I20260926 11:41:15.062287 16774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:15.048497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49396 (local address 127.6.96.62:40865)
0926 11:41:15.048888 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:15.048907 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:15.048949 (+ 42us) server_negotiation.cc:407] Connection header received
0926 11:41:15.049692 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:15.049711 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:15.050030 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:15.050314 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:15.051169 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.052155 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.053324 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.053804 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.053919 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:15.056601 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:41:15.056607 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:41:15.056626 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:41:15.056658 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:41:15.058463 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:41:15.058987 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:41:15.058993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:41:15.058999 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:41:15.059131 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:41:15.059447 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:41:15.059450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:41:15.059452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:41:15.059978 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:41:15.060179 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:15.061009 (+ 830us) server_negotiation.cc:299] Negotiation successful
0926 11:41:15.061313 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":114,"threads_started":1}
I20260926 11:41:15.063186 16783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:41:15.068928 16783 catalog_manager.cc:1347] Generated new cluster ID: 598ed03f26e44b8ba4f53b80012e4f4e
I20260926 11:41:15.069005 16783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:41:15.082571 16783 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:41:15.083565 16783 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:41:15.091943 16783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbb5238b828b442c887c28a932e8511e: Generated new TSK 0
I20260926 11:41:15.092729 16783 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.96.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
I20260926 11:41:15.127624 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:40865
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:41771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:41:15.234262 16797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:41:15.234515 16797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:41:15.234558 16797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:41:15.238545 16797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:41:15.238615 16797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:41:15.238672 16797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:41:15.243929 16797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:41771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:40865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:41:15.244963 16797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:41:15.245966 16797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:41:15.255200 16805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:15.255293 16797 server_base.cc:1034] running on GCE node
W20260926 11:41:15.255216 16802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:41:15.255216 16803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:15.255888 16797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:41:15.256498 16797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:41:15.257647 16797 hybrid_clock.cc:648] HybridClock initialized: now 1790422875257624 us; error 34 us; skew 500 ppm
Sep 26 11:41:15 dist-test-slave-xxxx krb5kdc[16690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422875, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:41:15.260586 16797 init.cc:377] Logged in from keytab as kudu/127.6.96.1@KRBTEST.COM (short username kudu)
I20260926 11:41:15.261626 16797 webserver.cc:460] Webserver started at http://127.6.96.1:32893/ using document root <none> and password file <none>
I20260926 11:41:15.262192 16797 fs_manager.cc:359] Metadata directory not provided
I20260926 11:41:15.262259 16797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:41:15.262449 16797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:41:15.264580 16797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "cbb09b9430a146b0aff1d73c09a9535c"
format_stamp: "Formatted at 2026-09-26 11:41:15 on dist-test-slave-xxxx"
server_key: "53973a75175d88ce7c81f666672ce171"
server_key_iv: "b4feb5016e2d165210e461d60f13ced4"
server_key_version: "encryptionkey@0"
I20260926 11:41:15.265146 16797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "cbb09b9430a146b0aff1d73c09a9535c"
format_stamp: "Formatted at 2026-09-26 11:41:15 on dist-test-slave-xxxx"
server_key: "53973a75175d88ce7c81f666672ce171"
server_key_iv: "b4feb5016e2d165210e461d60f13ced4"
server_key_version: "encryptionkey@0"
I20260926 11:41:15.268944 16797 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260926 11:41:15.271462 16812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:15.272708 16797 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:41:15.272818 16797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "cbb09b9430a146b0aff1d73c09a9535c"
format_stamp: "Formatted at 2026-09-26 11:41:15 on dist-test-slave-xxxx"
server_key: "53973a75175d88ce7c81f666672ce171"
server_key_iv: "b4feb5016e2d165210e461d60f13ced4"
server_key_version: "encryptionkey@0"
I20260926 11:41:15.272923 16797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:41:15.296625 16797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:41:15.299510 16797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:41:15.299690 16797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:41:15.300252 16797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:41:15.301038 16797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:41:15.301083 16797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:15.301136 16797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:41:15.301163 16797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:15.309162 16797 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:33535
I20260926 11:41:15.309217 16925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:33535 every 8 connection(s)
I20260926 11:41:15.310030 16797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:41:15.313748 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 16797
I20260926 11:41:15.313871 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:41:15.314164 6528 external_mini_cluster.cc:1428] Setting key 79bd105f3d77a2e456abdc4c4d06cb5b
Sep 26 11:41:15 dist-test-slave-xxxx krb5kdc[16690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422875, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.96.2@KRBTEST.COM; defaulting to no policy
I20260926 11:41:15.323928 16774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:15.311842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:54353 (local address 127.6.96.62:40865)
0926 11:41:15.311966 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:15.311969 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:15.312793 (+ 824us) server_negotiation.cc:407] Connection header received
0926 11:41:15.314262 (+ 1469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:15.314265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:15.314313 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:15.314385 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:15.315722 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.316236 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.317557 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.317916 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.318014 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:15.320539 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:41:15.320544 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:41:15.320547 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:41:15.320577 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:41:15.322048 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:41:15.322486 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:41:15.322490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:41:15.322492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:41:15.322579 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:41:15.322913 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:41:15.322916 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:41:15.322918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:41:15.323152 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:41:15.323252 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:15.323706 (+ 454us) server_negotiation.cc:299] Negotiation successful
0926 11:41:15.323808 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 11:41:15.324745 16928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:15.312227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:40865 (local address 127.6.96.1:54353)
0926 11:41:15.312646 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:15.312687 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:15.314017 (+ 1330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:15.314576 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:15.314584 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:15.315013 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:15.315584 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:15.315605 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.316373 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.316377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:15.317457 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:15.317466 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.318050 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.318053 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:15.318216 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:15.318745 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:15.318761 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:15.320376 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:41:15.322148 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:41:15.322153 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:41:15.322165 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:41:15.322394 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:41:15.322675 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:41:15.322678 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:41:15.322680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:41:15.322810 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:41:15.323243 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:41:15.323248 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:41:15.323470 (+ 222us) client_negotiation.cc:769] Sending connection context
0926 11:41:15.323623 (+ 153us) client_negotiation.cc:240] Negotiation successful
0926 11:41:15.323866 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":101,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.2@KRBTEST.COM" created.
I20260926 11:41:15.325911 16926 heartbeater.cc:347] Connected to a master server at 127.6.96.62:40865
I20260926 11:41:15.326164 16926 heartbeater.cc:464] Registering TS with master...
I20260926 11:41:15.326711 16926 heartbeater.cc:511] Master 127.6.96.62:40865 requested a full tablet report, sending...
I20260926 11:41:15.328459 16735 ts_manager.cc:194] Registered new tserver with Master: cbb09b9430a146b0aff1d73c09a9535c (127.6.96.1:33535)
I20260926 11:41:15.329972 16735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.1@KRBTEST.COM'} at 127.6.96.1:54353
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
I20260926 11:41:15.341742 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:40865
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:41771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:41:15.449962 16931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:41:15.450233 16931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:41:15.450276 16931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:41:15.454303 16931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:41:15.454362 16931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:41:15.454420 16931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:41:15.459594 16931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:41771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:40865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:41:15.460587 16931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:41:15.461524 16931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:41:15.471205 16936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:41:15.471192 16939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:15.471261 16931 server_base.cc:1034] running on GCE node
W20260926 11:41:15.471192 16937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:15.471823 16931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:41:15.472455 16931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:41:15.473649 16931 hybrid_clock.cc:648] HybridClock initialized: now 1790422875473616 us; error 47 us; skew 500 ppm
Sep 26 11:41:15 dist-test-slave-xxxx krb5kdc[16690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422875, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:41:15.476378 16931 init.cc:377] Logged in from keytab as kudu/127.6.96.2@KRBTEST.COM (short username kudu)
I20260926 11:41:15.477314 16931 webserver.cc:460] Webserver started at http://127.6.96.2:39925/ using document root <none> and password file <none>
I20260926 11:41:15.477882 16931 fs_manager.cc:359] Metadata directory not provided
I20260926 11:41:15.477941 16931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:41:15.478127 16931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:41:15.480257 16931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "02eaf43635ce40e9b8610e22a1614964"
format_stamp: "Formatted at 2026-09-26 11:41:15 on dist-test-slave-xxxx"
server_key: "ec9a43f0424dfb3a7852cac8f315f7df"
server_key_iv: "b79da9c060e7dbc96bc1a66d34d9d1df"
server_key_version: "encryptionkey@0"
I20260926 11:41:15.480795 16931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "02eaf43635ce40e9b8610e22a1614964"
format_stamp: "Formatted at 2026-09-26 11:41:15 on dist-test-slave-xxxx"
server_key: "ec9a43f0424dfb3a7852cac8f315f7df"
server_key_iv: "b79da9c060e7dbc96bc1a66d34d9d1df"
server_key_version: "encryptionkey@0"
I20260926 11:41:15.484630 16931 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 11:41:15.487118 16946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:15.488374 16931 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 11:41:15.488472 16931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "02eaf43635ce40e9b8610e22a1614964"
format_stamp: "Formatted at 2026-09-26 11:41:15 on dist-test-slave-xxxx"
server_key: "ec9a43f0424dfb3a7852cac8f315f7df"
server_key_iv: "b79da9c060e7dbc96bc1a66d34d9d1df"
server_key_version: "encryptionkey@0"
I20260926 11:41:15.488565 16931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:41:15.516681 16931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:41:15.519629 16931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:41:15.519793 16931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:41:15.520328 16931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:41:15.521128 16931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:41:15.521174 16931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:15.521214 16931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:41:15.521234 16931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:15.529253 16931 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:35765
I20260926 11:41:15.529275 17059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:35765 every 8 connection(s)
I20260926 11:41:15.530107 16931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:41:15.537827 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 16931
I20260926 11:41:15.537951 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:41:15.538254 6528 external_mini_cluster.cc:1428] Setting key c6b069da6867d1105278e0e2d93fddf5
Sep 26 11:41:15 dist-test-slave-xxxx krb5kdc[16690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422875, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:41:15.543215 16774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:15.531895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:45619 (local address 127.6.96.62:40865)
0926 11:41:15.532020 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:15.532024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:15.532821 (+ 797us) server_negotiation.cc:407] Connection header received
0926 11:41:15.534128 (+ 1307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:15.534132 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:15.534186 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:15.534272 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:15.535455 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.535969 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.537088 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.537299 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.537385 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:15.539938 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:41:15.539941 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:41:15.539943 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:41:15.539966 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:41:15.541208 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:41:15.541754 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:41:15.541756 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:41:15.541758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:41:15.541827 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:41:15.542158 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:41:15.542160 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:41:15.542162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:41:15.542366 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:41:15.542458 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:15.542935 (+ 477us) server_negotiation.cc:299] Negotiation successful
0926 11:41:15.543049 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 11:41:15.544015 17062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:15.532266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:40865 (local address 127.6.96.2:45619)
0926 11:41:15.532686 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:15.532728 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:15.533926 (+ 1198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:15.534436 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:15.534442 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:15.534801 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:15.535314 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:15.535328 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.536084 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.536087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:15.536962 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:15.536972 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.537433 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.537437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:15.537632 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:15.538235 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:15.538255 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:15.539784 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:41:15.541316 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:41:15.541322 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:41:15.541337 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:41:15.541666 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:41:15.541900 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:41:15.541903 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:41:15.541905 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:41:15.542071 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:41:15.542448 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:41:15.542453 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:41:15.542710 (+ 257us) client_negotiation.cc:769] Sending connection context
0926 11:41:15.542895 (+ 185us) client_negotiation.cc:240] Negotiation successful
0926 11:41:15.543151 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":101,"threads_started":1}
I20260926 11:41:15.545086 17060 heartbeater.cc:347] Connected to a master server at 127.6.96.62:40865
I20260926 11:41:15.545329 17060 heartbeater.cc:464] Registering TS with master...
I20260926 11:41:15.545907 17060 heartbeater.cc:511] Master 127.6.96.62:40865 requested a full tablet report, sending...
I20260926 11:41:15.546927 16735 ts_manager.cc:194] Registered new tserver with Master: 02eaf43635ce40e9b8610e22a1614964 (127.6.96.2:35765)
WARNING: no policy specified for kudu/127.6.96.3@KRBTEST.COM; defaulting to no policy
I20260926 11:41:15.547497 16735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.2@KRBTEST.COM'} at 127.6.96.2:45619
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
I20260926 11:41:15.563755 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:40865
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:41771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:41:15.670653 17065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:41:15.670951 17065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:41:15.671015 17065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:41:15.675159 17065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:41:15.675228 17065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:41:15.675308 17065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:41:15.680806 17065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:41771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:40865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:41:15.681869 17065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:41:15.682920 17065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:41:15.692389 17071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:15.692535 17065 server_base.cc:1034] running on GCE node
W20260926 11:41:15.692452 17073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:41:15.692397 17070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:15.693135 17065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:41:15.693789 17065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:41:15.694895 17065 hybrid_clock.cc:648] HybridClock initialized: now 1790422875694874 us; error 36 us; skew 500 ppm
Sep 26 11:41:15 dist-test-slave-xxxx krb5kdc[16690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422875, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:41:15.697448 17065 init.cc:377] Logged in from keytab as kudu/127.6.96.3@KRBTEST.COM (short username kudu)
I20260926 11:41:15.698345 17065 webserver.cc:460] Webserver started at http://127.6.96.3:34037/ using document root <none> and password file <none>
I20260926 11:41:15.698916 17065 fs_manager.cc:359] Metadata directory not provided
I20260926 11:41:15.698982 17065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:41:15.699179 17065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:41:15.701143 17065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "7a660fd52e5247ccbbfe17ceb9e375a8"
format_stamp: "Formatted at 2026-09-26 11:41:15 on dist-test-slave-xxxx"
server_key: "42e8f318370aba7eb8f5dccce05ad1de"
server_key_iv: "dc87edf6dc45d27ed3695a6e04b1938f"
server_key_version: "encryptionkey@0"
I20260926 11:41:15.701663 17065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "7a660fd52e5247ccbbfe17ceb9e375a8"
format_stamp: "Formatted at 2026-09-26 11:41:15 on dist-test-slave-xxxx"
server_key: "42e8f318370aba7eb8f5dccce05ad1de"
server_key_iv: "dc87edf6dc45d27ed3695a6e04b1938f"
server_key_version: "encryptionkey@0"
I20260926 11:41:15.705220 17065 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260926 11:41:15.707584 17080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:15.708741 17065 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:41:15.708844 17065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "7a660fd52e5247ccbbfe17ceb9e375a8"
format_stamp: "Formatted at 2026-09-26 11:41:15 on dist-test-slave-xxxx"
server_key: "42e8f318370aba7eb8f5dccce05ad1de"
server_key_iv: "dc87edf6dc45d27ed3695a6e04b1938f"
server_key_version: "encryptionkey@0"
I20260926 11:41:15.708935 17065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:41:15.725131 17065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:41:15.727782 17065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:41:15.727952 17065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:41:15.728504 17065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:41:15.729282 17065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:41:15.729328 17065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:15.729379 17065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:41:15.729408 17065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:15.737483 17065 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:38235
I20260926 11:41:15.737517 17193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:38235 every 8 connection(s)
I20260926 11:41:15.738300 17065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:41:15.739051 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 17065
I20260926 11:41:15.739161 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:41:15.739454 6528 external_mini_cluster.cc:1428] Setting key 68c2d9321d20905492dff6e6ca70fbf4
Sep 26 11:41:15 dist-test-slave-xxxx krb5kdc[16690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422875, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:41:15.750725 16774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:15.740073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:46199 (local address 127.6.96.62:40865)
0926 11:41:15.740184 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:15.740189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:15.741167 (+ 978us) server_negotiation.cc:407] Connection header received
0926 11:41:15.742520 (+ 1353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:15.742522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:15.742565 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:15.742627 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:15.743688 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.744212 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.745384 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.745589 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.745664 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:15.747882 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:41:15.747886 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:41:15.747888 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:41:15.747911 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:41:15.749148 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:41:15.749583 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:41:15.749587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:41:15.749588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:41:15.749665 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:41:15.749902 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:41:15.749904 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:41:15.749905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:41:15.750055 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:41:15.750118 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:15.750513 (+ 395us) server_negotiation.cc:299] Negotiation successful
0926 11:41:15.750602 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 11:41:15.751529 17196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:15.740534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:40865 (local address 127.6.96.3:46199)
0926 11:41:15.741021 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:15.741078 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:15.742339 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:15.742759 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:15.742765 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:15.743117 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:15.743573 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:15.743587 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.744349 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.744352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:15.745281 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:15.745291 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.745699 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.745702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:15.745835 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:15.746329 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:15.746344 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:15.747730 (+ 1386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:41:15.749249 (+ 1519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:41:15.749253 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:41:15.749264 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:41:15.749514 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:41:15.749730 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:41:15.749732 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:41:15.749733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:41:15.749849 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:41:15.750122 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:41:15.750128 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:41:15.750336 (+ 208us) client_negotiation.cc:769] Sending connection context
0926 11:41:15.750481 (+ 145us) client_negotiation.cc:240] Negotiation successful
0926 11:41:15.750732 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":123,"threads_started":1}
I20260926 11:41:15.752532 17194 heartbeater.cc:347] Connected to a master server at 127.6.96.62:40865
I20260926 11:41:15.752767 17194 heartbeater.cc:464] Registering TS with master...
I20260926 11:41:15.753259 17194 heartbeater.cc:511] Master 127.6.96.62:40865 requested a full tablet report, sending...
I20260926 11:41:15.754134 16734 ts_manager.cc:194] Registered new tserver with Master: 7a660fd52e5247ccbbfe17ceb9e375a8 (127.6.96.3:38235)
I20260926 11:41:15.754678 16734 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.3@KRBTEST.COM'} at 127.6.96.3:46199
I20260926 11:41:15.764573 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 11:41:15.773048 16774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:15.766373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50116 (local address 127.6.96.62:40865)
0926 11:41:15.766495 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:15.766499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:15.766653 (+ 154us) server_negotiation.cc:407] Connection header received
0926 11:41:15.766770 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:15.766773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:15.766816 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:15.766909 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:15.767589 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.768096 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.768899 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.769106 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.769190 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:15.770388 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:41:15.770392 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:41:15.770395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:41:15.770420 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:41:15.771565 (+ 1145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:41:15.771951 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:41:15.771954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:41:15.771956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:41:15.772027 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:41:15.772263 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:41:15.772266 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:41:15.772268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:41:15.772470 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:41:15.772581 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:15.772802 (+ 221us) server_negotiation.cc:299] Negotiation successful
0926 11:41:15.772906 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 11:41:15.775600 16734 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 11:41:15.778576 16734 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:41:15.792264 17210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:15.787476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34294 (local address 127.6.96.3:38235)
0926 11:41:15.787789 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:15.787800 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:15.787820 (+ 20us) server_negotiation.cc:407] Connection header received
0926 11:41:15.787890 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:15.787896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:15.788090 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:15.788462 (+ 372us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:41:15.788579 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.789378 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.790795 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.791616 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.791730 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:15.791875 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:15.792021 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 11:41:15.792119 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":122,"threads_started":1}
I20260926 11:41:15.792258 17206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:15.786602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:38235 (local address 127.0.0.1:34294)
0926 11:41:15.787153 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:15.787177 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:15.787281 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:15.788233 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:15.788237 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:15.788254 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:41:15.788346 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:15.788350 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.789533 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.789536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:15.790656 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:15.790676 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.791737 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.791741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:15.791841 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:15.791875 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 11:41:15.791983 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 11:41:15.792100 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"mutex_wait_us":36,"spinlock_wait_cycles":8064,"thread_start_us":87,"threads_started":1}
I20260926 11:41:15.792568 17207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:15.786413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42506 (local address 127.6.96.1:33535)
0926 11:41:15.787522 (+ 1109us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:15.787534 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:15.787553 (+ 19us) server_negotiation.cc:407] Connection header received
0926 11:41:15.787610 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:15.787614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:15.787773 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:15.787906 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:41:15.788196 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.789338 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.790896 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.791655 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.791734 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:15.791873 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:15.792341 (+ 468us) server_negotiation.cc:299] Negotiation successful
0926 11:41:15.792436 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1023,"thread_start_us":68,"threads_started":1}
I20260926 11:41:15.792279 17208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:15.786282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:33535 (local address 127.0.0.1:42506)
0926 11:41:15.787088 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:15.787138 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:15.787281 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:15.787910 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:15.787916 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:15.787937 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:41:15.788067 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:15.788074 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.789466 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.789469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:15.790748 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:15.790758 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.791961 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.791964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:15.792042 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:15.792059 (+ 17us) client_negotiation.cc:769] Sending connection context
0926 11:41:15.792127 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 11:41:15.792182 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":69,"spinlock_wait_cycles":35584,"thread_start_us":68,"threads_started":1}
I20260926 11:41:15.793125 17205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:15.786355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:35765 (local address 127.0.0.1:40588)
0926 11:41:15.787113 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:15.787148 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:15.787281 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:15.788694 (+ 1413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:15.788697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:15.788713 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:41:15.788787 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:15.788792 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.790117 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.790120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:15.791207 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:15.791217 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.792755 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.792759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:15.792844 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:15.792867 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 11:41:15.792942 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 11:41:15.793004 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"spinlock_wait_cycles":13312,"thread_start_us":92,"threads_started":1}
I20260926 11:41:15.793182 17209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:15.787083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40588 (local address 127.6.96.2:35765)
0926 11:41:15.788278 (+ 1195us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:15.788292 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:15.788314 (+ 22us) server_negotiation.cc:407] Connection header received
0926 11:41:15.788382 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:15.788387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:15.788558 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:15.788703 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:41:15.788910 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.789977 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.791326 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.792066 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.792171 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:15.792291 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:15.792986 (+ 695us) server_negotiation.cc:299] Negotiation successful
0926 11:41:15.793072 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1090,"thread_start_us":114,"threads_started":1}
I20260926 11:41:15.795780 16860 tablet_service.cc:1467] Processing CreateTablet for tablet cf7c677f3ced4999a75afa4d9e5607d9 (DEFAULT_TABLE table=test-table [id=7a63757b21e7432c94900d53369b41c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:41:15.795795 17128 tablet_service.cc:1467] Processing CreateTablet for tablet cf7c677f3ced4999a75afa4d9e5607d9 (DEFAULT_TABLE table=test-table [id=7a63757b21e7432c94900d53369b41c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:41:15.795794 16994 tablet_service.cc:1467] Processing CreateTablet for tablet cf7c677f3ced4999a75afa4d9e5607d9 (DEFAULT_TABLE table=test-table [id=7a63757b21e7432c94900d53369b41c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:41:15.796726 16860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf7c677f3ced4999a75afa4d9e5607d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:41:15.796765 16994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf7c677f3ced4999a75afa4d9e5607d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:41:15.796774 17128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf7c677f3ced4999a75afa4d9e5607d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:41:15.804366 17211 tablet_bootstrap.cc:492] T cf7c677f3ced4999a75afa4d9e5607d9 P cbb09b9430a146b0aff1d73c09a9535c: Bootstrap starting.
I20260926 11:41:15.804700 17212 tablet_bootstrap.cc:492] T cf7c677f3ced4999a75afa4d9e5607d9 P 7a660fd52e5247ccbbfe17ceb9e375a8: Bootstrap starting.
I20260926 11:41:15.806567 17211 tablet_bootstrap.cc:654] T cf7c677f3ced4999a75afa4d9e5607d9 P cbb09b9430a146b0aff1d73c09a9535c: Neither blocks nor log segments found. Creating new log.
I20260926 11:41:15.806998 17212 tablet_bootstrap.cc:654] T cf7c677f3ced4999a75afa4d9e5607d9 P 7a660fd52e5247ccbbfe17ceb9e375a8: Neither blocks nor log segments found. Creating new log.
I20260926 11:41:15.807727 17211 log.cc:824] T cf7c677f3ced4999a75afa4d9e5607d9 P cbb09b9430a146b0aff1d73c09a9535c: Log is configured to *not* fsync() on all Append() calls
I20260926 11:41:15.807729 17212 log.cc:824] T cf7c677f3ced4999a75afa4d9e5607d9 P 7a660fd52e5247ccbbfe17ceb9e375a8: Log is configured to *not* fsync() on all Append() calls
I20260926 11:41:15.808488 17213 tablet_bootstrap.cc:492] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964: Bootstrap starting.
I20260926 11:41:15.809814 17212 tablet_bootstrap.cc:492] T cf7c677f3ced4999a75afa4d9e5607d9 P 7a660fd52e5247ccbbfe17ceb9e375a8: No bootstrap required, opened a new log
I20260926 11:41:15.810021 17212 ts_tablet_manager.cc:1397] T cf7c677f3ced4999a75afa4d9e5607d9 P 7a660fd52e5247ccbbfe17ceb9e375a8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 11:41:15.810534 17211 tablet_bootstrap.cc:492] T cf7c677f3ced4999a75afa4d9e5607d9 P cbb09b9430a146b0aff1d73c09a9535c: No bootstrap required, opened a new log
I20260926 11:41:15.810727 17211 ts_tablet_manager.cc:1397] T cf7c677f3ced4999a75afa4d9e5607d9 P cbb09b9430a146b0aff1d73c09a9535c: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260926 11:41:15.811430 17213 tablet_bootstrap.cc:654] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964: Neither blocks nor log segments found. Creating new log.
I20260926 11:41:15.812255 17213 log.cc:824] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964: Log is configured to *not* fsync() on all Append() calls
I20260926 11:41:15.814090 17213 tablet_bootstrap.cc:492] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964: No bootstrap required, opened a new log
I20260926 11:41:15.814276 17213 ts_tablet_manager.cc:1397] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 11:41:15.814077 17212 raft_consensus.cc:348] T cf7c677f3ced4999a75afa4d9e5607d9 P 7a660fd52e5247ccbbfe17ceb9e375a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb09b9430a146b0aff1d73c09a9535c" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 33535 } } peers { permanent_uuid: "02eaf43635ce40e9b8610e22a1614964" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35765 } } peers { permanent_uuid: "7a660fd52e5247ccbbfe17ceb9e375a8" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 38235 } }
I20260926 11:41:15.814352 17212 raft_consensus.cc:374] T cf7c677f3ced4999a75afa4d9e5607d9 P 7a660fd52e5247ccbbfe17ceb9e375a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:41:15.814407 17212 raft_consensus.cc:729] T cf7c677f3ced4999a75afa4d9e5607d9 P 7a660fd52e5247ccbbfe17ceb9e375a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a660fd52e5247ccbbfe17ceb9e375a8, State: Initialized, Role: FOLLOWER
I20260926 11:41:15.814504 17211 raft_consensus.cc:348] T cf7c677f3ced4999a75afa4d9e5607d9 P cbb09b9430a146b0aff1d73c09a9535c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb09b9430a146b0aff1d73c09a9535c" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 33535 } } peers { permanent_uuid: "02eaf43635ce40e9b8610e22a1614964" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35765 } } peers { permanent_uuid: "7a660fd52e5247ccbbfe17ceb9e375a8" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 38235 } }
I20260926 11:41:15.814741 17211 raft_consensus.cc:374] T cf7c677f3ced4999a75afa4d9e5607d9 P cbb09b9430a146b0aff1d73c09a9535c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:41:15.814801 17211 raft_consensus.cc:729] T cf7c677f3ced4999a75afa4d9e5607d9 P cbb09b9430a146b0aff1d73c09a9535c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbb09b9430a146b0aff1d73c09a9535c, State: Initialized, Role: FOLLOWER
I20260926 11:41:15.814909 17212 consensus_queue.cc:260] T cf7c677f3ced4999a75afa4d9e5607d9 P 7a660fd52e5247ccbbfe17ceb9e375a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb09b9430a146b0aff1d73c09a9535c" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 33535 } } peers { permanent_uuid: "02eaf43635ce40e9b8610e22a1614964" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35765 } } peers { permanent_uuid: "7a660fd52e5247ccbbfe17ceb9e375a8" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 38235 } }
I20260926 11:41:15.815276 17211 consensus_queue.cc:260] T cf7c677f3ced4999a75afa4d9e5607d9 P cbb09b9430a146b0aff1d73c09a9535c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb09b9430a146b0aff1d73c09a9535c" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 33535 } } peers { permanent_uuid: "02eaf43635ce40e9b8610e22a1614964" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35765 } } peers { permanent_uuid: "7a660fd52e5247ccbbfe17ceb9e375a8" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 38235 } }
I20260926 11:41:15.815673 17194 heartbeater.cc:503] Master 127.6.96.62:40865 was elected leader, sending a full tablet report...
I20260926 11:41:15.816303 17211 ts_tablet_manager.cc:1428] T cf7c677f3ced4999a75afa4d9e5607d9 P cbb09b9430a146b0aff1d73c09a9535c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 11:41:15.816627 16926 heartbeater.cc:503] Master 127.6.96.62:40865 was elected leader, sending a full tablet report...
I20260926 11:41:15.817912 17212 ts_tablet_manager.cc:1428] T cf7c677f3ced4999a75afa4d9e5607d9 P 7a660fd52e5247ccbbfe17ceb9e375a8: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260926 11:41:15.818280 17213 raft_consensus.cc:348] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb09b9430a146b0aff1d73c09a9535c" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 33535 } } peers { permanent_uuid: "02eaf43635ce40e9b8610e22a1614964" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35765 } } peers { permanent_uuid: "7a660fd52e5247ccbbfe17ceb9e375a8" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 38235 } }
I20260926 11:41:15.818545 17213 raft_consensus.cc:374] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:41:15.818605 17213 raft_consensus.cc:729] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02eaf43635ce40e9b8610e22a1614964, State: Initialized, Role: FOLLOWER
I20260926 11:41:15.819114 17213 consensus_queue.cc:260] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb09b9430a146b0aff1d73c09a9535c" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 33535 } } peers { permanent_uuid: "02eaf43635ce40e9b8610e22a1614964" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35765 } } peers { permanent_uuid: "7a660fd52e5247ccbbfe17ceb9e375a8" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 38235 } }
I20260926 11:41:15.819787 17060 heartbeater.cc:503] Master 127.6.96.62:40865 was elected leader, sending a full tablet report...
I20260926 11:41:15.820065 17213 ts_tablet_manager.cc:1428] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 11:41:15.888350 17219 raft_consensus.cc:482] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:41:15.888540 17219 raft_consensus.cc:504] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb09b9430a146b0aff1d73c09a9535c" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 33535 } } peers { permanent_uuid: "02eaf43635ce40e9b8610e22a1614964" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35765 } } peers { permanent_uuid: "7a660fd52e5247ccbbfe17ceb9e375a8" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 38235 } }
I20260926 11:41:15.889662 17219 leader_election.cc:288] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cbb09b9430a146b0aff1d73c09a9535c (127.6.96.1:33535), 7a660fd52e5247ccbbfe17ceb9e375a8 (127.6.96.3:38235)
I20260926 11:41:15.894443 17210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:15.889986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:57893 (local address 127.6.96.3:38235)
0926 11:41:15.890139 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:15.890143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:15.890394 (+ 251us) server_negotiation.cc:407] Connection header received
0926 11:41:15.890481 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:15.890484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:15.890530 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:15.890597 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:41:15.890817 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.891570 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.893278 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.893876 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.893981 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:15.894046 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:15.894245 (+ 199us) server_negotiation.cc:299] Negotiation successful
0926 11:41:15.894313 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 11:41:15.894443 17207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:15.889983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:40275 (local address 127.6.96.1:33535)
0926 11:41:15.890114 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:15.890119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:15.890226 (+ 107us) server_negotiation.cc:407] Connection header received
0926 11:41:15.890356 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:15.890358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:15.890411 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:15.890495 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:41:15.890740 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.891571 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.893278 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.893876 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.893981 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:15.894046 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:15.894245 (+ 199us) server_negotiation.cc:299] Negotiation successful
0926 11:41:15.894313 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 11:41:15.894490 17062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:15.889913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:33535 (local address 127.6.96.2:40275)
0926 11:41:15.890131 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:15.890146 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:15.890260 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:15.890510 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:15.890512 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:15.890525 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:41:15.890635 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:15.890642 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.891690 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.891693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:15.893154 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:15.893166 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.893989 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.893993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:15.894145 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:15.894162 (+ 17us) client_negotiation.cc:769] Sending connection context
0926 11:41:15.894223 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 11:41:15.894346 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":186}
I20260926 11:41:15.894497 17220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:15.889913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:38235 (local address 127.6.96.2:57893)
0926 11:41:15.890312 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:15.890323 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:15.890404 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:15.890621 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:15.890624 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:15.890637 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:41:15.890732 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:15.890738 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.891690 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.891693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:15.893154 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:15.893166 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.893989 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.893993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:15.894119 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:15.894145 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 11:41:15.894216 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 11:41:15.894340 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":162,"thread_start_us":116,"threads_started":1}
I20260926 11:41:15.895190 17148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf7c677f3ced4999a75afa4d9e5607d9" candidate_uuid: "02eaf43635ce40e9b8610e22a1614964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a660fd52e5247ccbbfe17ceb9e375a8" is_pre_election: true
I20260926 11:41:15.895205 16880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf7c677f3ced4999a75afa4d9e5607d9" candidate_uuid: "02eaf43635ce40e9b8610e22a1614964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbb09b9430a146b0aff1d73c09a9535c" is_pre_election: true
I20260926 11:41:15.895469 17148 raft_consensus.cc:2454] T cf7c677f3ced4999a75afa4d9e5607d9 P 7a660fd52e5247ccbbfe17ceb9e375a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02eaf43635ce40e9b8610e22a1614964 in term 0.
I20260926 11:41:15.895469 16880 raft_consensus.cc:2454] T cf7c677f3ced4999a75afa4d9e5607d9 P cbb09b9430a146b0aff1d73c09a9535c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02eaf43635ce40e9b8610e22a1614964 in term 0.
I20260926 11:41:15.895866 16950 leader_election.cc:302] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02eaf43635ce40e9b8610e22a1614964, 7a660fd52e5247ccbbfe17ceb9e375a8; no voters:
I20260926 11:41:15.896133 17219 raft_consensus.cc:2781] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:41:15.896214 17219 raft_consensus.cc:482] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:41:15.896261 17219 raft_consensus.cc:3037] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:41:15.897204 17219 raft_consensus.cc:504] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb09b9430a146b0aff1d73c09a9535c" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 33535 } } peers { permanent_uuid: "02eaf43635ce40e9b8610e22a1614964" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35765 } } peers { permanent_uuid: "7a660fd52e5247ccbbfe17ceb9e375a8" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 38235 } }
I20260926 11:41:15.897504 17219 leader_election.cc:288] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964 [CANDIDATE]: Term 1 election: Requested vote from peers cbb09b9430a146b0aff1d73c09a9535c (127.6.96.1:33535), 7a660fd52e5247ccbbfe17ceb9e375a8 (127.6.96.3:38235)
I20260926 11:41:15.897831 16880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf7c677f3ced4999a75afa4d9e5607d9" candidate_uuid: "02eaf43635ce40e9b8610e22a1614964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbb09b9430a146b0aff1d73c09a9535c"
I20260926 11:41:15.897851 17148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf7c677f3ced4999a75afa4d9e5607d9" candidate_uuid: "02eaf43635ce40e9b8610e22a1614964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a660fd52e5247ccbbfe17ceb9e375a8"
I20260926 11:41:15.897939 17148 raft_consensus.cc:3037] T cf7c677f3ced4999a75afa4d9e5607d9 P 7a660fd52e5247ccbbfe17ceb9e375a8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:41:15.897943 16880 raft_consensus.cc:3037] T cf7c677f3ced4999a75afa4d9e5607d9 P cbb09b9430a146b0aff1d73c09a9535c [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:41:15.898923 16880 raft_consensus.cc:2454] T cf7c677f3ced4999a75afa4d9e5607d9 P cbb09b9430a146b0aff1d73c09a9535c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02eaf43635ce40e9b8610e22a1614964 in term 1.
I20260926 11:41:15.898975 17148 raft_consensus.cc:2454] T cf7c677f3ced4999a75afa4d9e5607d9 P 7a660fd52e5247ccbbfe17ceb9e375a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02eaf43635ce40e9b8610e22a1614964 in term 1.
I20260926 11:41:15.899200 16948 leader_election.cc:302] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02eaf43635ce40e9b8610e22a1614964, cbb09b9430a146b0aff1d73c09a9535c; no voters:
I20260926 11:41:15.899354 17219 raft_consensus.cc:2781] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:41:15.899551 17219 raft_consensus.cc:686] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964 [term 1 LEADER]: Becoming Leader. State: Replica: 02eaf43635ce40e9b8610e22a1614964, State: Running, Role: LEADER
I20260926 11:41:15.899848 17219 consensus_queue.cc:237] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb09b9430a146b0aff1d73c09a9535c" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 33535 } } peers { permanent_uuid: "02eaf43635ce40e9b8610e22a1614964" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35765 } } peers { permanent_uuid: "7a660fd52e5247ccbbfe17ceb9e375a8" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 38235 } }
I20260926 11:41:15.902733 16734 catalog_manager.cc:5495] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964 reported cstate change: term changed from 0 to 1, leader changed from <none> to 02eaf43635ce40e9b8610e22a1614964 (127.6.96.2). New cstate: current_term: 1 leader_uuid: "02eaf43635ce40e9b8610e22a1614964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbb09b9430a146b0aff1d73c09a9535c" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 33535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02eaf43635ce40e9b8610e22a1614964" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a660fd52e5247ccbbfe17ceb9e375a8" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 38235 } health_report { overall_health: UNKNOWN } } }
I20260926 11:41:15.915004 17209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:15.911631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40598 (local address 127.6.96.2:35765)
0926 11:41:15.911732 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:15.911735 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:15.911746 (+ 11us) server_negotiation.cc:407] Connection header received
0926 11:41:15.911897 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:15.911900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:15.911953 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:15.912040 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:41:15.912263 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.912794 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.913628 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:15.913835 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:15.913925 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:15.914205 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:41:15.914609 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:41:15.914715 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:15.914783 (+ 68us) server_negotiation.cc:299] Negotiation successful
0926 11:41:15.914858 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 11:41:15.920532 16880 raft_consensus.cc:1261] T cf7c677f3ced4999a75afa4d9e5607d9 P cbb09b9430a146b0aff1d73c09a9535c [term 1 FOLLOWER]: Refusing update from remote peer 02eaf43635ce40e9b8610e22a1614964: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:41:15.920532 17148 raft_consensus.cc:1261] T cf7c677f3ced4999a75afa4d9e5607d9 P 7a660fd52e5247ccbbfe17ceb9e375a8 [term 1 FOLLOWER]: Refusing update from remote peer 02eaf43635ce40e9b8610e22a1614964: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:41:15.921204 17221 consensus_queue.cc:1035] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbb09b9430a146b0aff1d73c09a9535c" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 33535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:41:15.921366 17219 consensus_queue.cc:1035] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a660fd52e5247ccbbfe17ceb9e375a8" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 38235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:41:15.929729 17226 mvcc.cc:205] Tried to move back new op lower bound from 7333572099763793920 to 7333572099688099840. Current Snapshot: MvccSnapshot[applied={T|T < 7333572099763793920}]
I20260926 11:41:15.931550 17225 mvcc.cc:205] Tried to move back new op lower bound from 7333572099763793920 to 7333572099688099840. Current Snapshot: MvccSnapshot[applied={T|T < 7333572099763793920}]
I20260926 11:41:15.937245 16735 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:41:15.937392 16735 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:41:15.939502 16735 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bbb5238b828b442c887c28a932e8511e: Sending DeleteTablet for 3 replicas of tablet cf7c677f3ced4999a75afa4d9e5607d9
I20260926 11:41:15.940192 16860 tablet_service.cc:1514] Processing DeleteTablet for tablet cf7c677f3ced4999a75afa4d9e5607d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:41:15 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:42506
I20260926 11:41:15.940231 16994 tablet_service.cc:1514] Processing DeleteTablet for tablet cf7c677f3ced4999a75afa4d9e5607d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:41:15 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:40588
I20260926 11:41:15.940322 17128 tablet_service.cc:1514] Processing DeleteTablet for tablet cf7c677f3ced4999a75afa4d9e5607d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:41:15 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:34294
I20260926 11:41:15.940781 17234 tablet_replica.cc:330] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964: stopping tablet replica
I20260926 11:41:15.940809 17235 tablet_replica.cc:330] T cf7c677f3ced4999a75afa4d9e5607d9 P 7a660fd52e5247ccbbfe17ceb9e375a8: stopping tablet replica
I20260926 11:41:15.941095 17235 raft_consensus.cc:2229] T cf7c677f3ced4999a75afa4d9e5607d9 P 7a660fd52e5247ccbbfe17ceb9e375a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:41:15.941099 17234 raft_consensus.cc:2229] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964 [term 1 LEADER]: Raft consensus shutting down.
I20260926 11:41:15.941318 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 16797
I20260926 11:41:15.941519 17235 raft_consensus.cc:2258] T cf7c677f3ced4999a75afa4d9e5607d9 P 7a660fd52e5247ccbbfe17ceb9e375a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:41:15.942114 17234 raft_consensus.cc:2258] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:41:15.943579 17234 ts_tablet_manager.cc:1905] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:41:15.944756 17235 ts_tablet_manager.cc:1905] T cf7c677f3ced4999a75afa4d9e5607d9 P 7a660fd52e5247ccbbfe17ceb9e375a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:41:15.946399 17234 ts_tablet_manager.cc:1918] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:41:15.946557 17234 log.cc:1196] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-1/wal/wals/cf7c677f3ced4999a75afa4d9e5607d9
I20260926 11:41:15.946935 17234 ts_tablet_manager.cc:1939] T cf7c677f3ced4999a75afa4d9e5607d9 P 02eaf43635ce40e9b8610e22a1614964: Deleting consensus metadata
I20260926 11:41:15.947223 17235 ts_tablet_manager.cc:1918] T cf7c677f3ced4999a75afa4d9e5607d9 P 7a660fd52e5247ccbbfe17ceb9e375a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:41:15.947312 17235 log.cc:1196] T cf7c677f3ced4999a75afa4d9e5607d9 P 7a660fd52e5247ccbbfe17ceb9e375a8: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/cf7c677f3ced4999a75afa4d9e5607d9
I20260926 11:41:15.947598 17235 ts_tablet_manager.cc:1939] T cf7c677f3ced4999a75afa4d9e5607d9 P 7a660fd52e5247ccbbfe17ceb9e375a8: Deleting consensus metadata
I20260926 11:41:15.948279 16721 catalog_manager.cc:4841] TS 02eaf43635ce40e9b8610e22a1614964 (127.6.96.2:35765): tablet cf7c677f3ced4999a75afa4d9e5607d9 (table test-table [id=7a63757b21e7432c94900d53369b41c6]) successfully deleted
I20260926 11:41:15.948637 16722 catalog_manager.cc:4841] TS 7a660fd52e5247ccbbfe17ceb9e375a8 (127.6.96.3:38235): tablet cf7c677f3ced4999a75afa4d9e5607d9 (table test-table [id=7a63757b21e7432c94900d53369b41c6]) successfully deleted
W20260926 11:41:15.948786 16720 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.96.1:33535 (error 108)
I20260926 11:41:15.949426 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 16931
I20260926 11:41:15.949774 17205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:15.949481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:33535 (local address 127.0.0.1:42522)
0926 11:41:15.949635 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:15.949696 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.1:33535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260926 11:41:15.949913 16720 catalog_manager.cc:4584] TS cbb09b9430a146b0aff1d73c09a9535c (127.6.96.1:33535): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cf7c677f3ced4999a75afa4d9e5607d9: Network error: Client connection negotiation failed: client connection to 127.6.96.1:33535: connect: Connection refused (error 111)
I20260926 11:41:15.954471 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 17065
I20260926 11:41:15.958811 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 16703
2026-09-26T11:41:15Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2156 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:41:16 dist-test-slave-xxxx krb5kdc[17239](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:41:16 dist-test-slave-xxxx krb5kdc[17239](info): set up 2 sockets
Sep 26 11:41:16 dist-test-slave-xxxx krb5kdc[17239](info): commencing operation
krb5kdc: starting...
W20260926 11:41:16.919461 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:41:16 dist-test-slave-xxxx krb5kdc[17239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:41:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:41:17Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.96.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.96.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.96.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/rangeradmin_127.6.96.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.96.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/rangeradmin_127.6.96.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.96.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.96.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.96.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/rangerlookup_127.6.96.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.96.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/rangerlookup_127.6.96.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.96.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.96.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.96.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/HTTP_127.6.96.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.96.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/HTTP_127.6.96.20@KRBTEST.COM.keytab.
I20260926 11:41:17.081780 6528 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskBunNuW/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/postgres -l logfile start
2026-09-26 11:41:19.392 UTC [17269] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 11:41:19.392 UTC [17269] LOG: listening on IPv4 address "127.6.96.20", port 40909
2026-09-26 11:41:19.395 UTC [17269] LOG: listening on Unix socket "/tmp/.s.PGSQL.40909"
2026-09-26 11:41:19.421 UTC [17272] LOG: database system was shut down at 2026-09-26 11:41:18 UTC
2026-09-26 11:41:19.425 UTC [17269] LOG: database system is ready to accept connections
I20260926 11:41:20.287285 6528 mini_postgres.cc:96] Postgres bound to 40909
2026-09-26 11:41:20.293 UTC [17280] FATAL: database "slave" does not exist
127.6.96.20:40909 - accepting connections
I20260926 11:41:20.293607 6528 mini_ranger.cc:161] Starting Ranger...
I20260926 11:41:20.307512 6528 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 11:41:20.743884 6528 mini_ranger.cc:87] Created ranger Postgres database
I20260926 11:41:20.743973 6528 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ranger-admin
2026-09-26 11:41:21,020 [I] DB FLAVOR :POSTGRES
2026-09-26 11:41:21,020 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 11:41:21,020 [I] Checking connection..
2026-09-26 11:41:21,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:40909/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 11:41:21,275 [I] Checking connection passed.
2026-09-26 11:41:21,276 [I] --------- Verifying version history table ---------
2026-09-26 11:41:21,276 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:40909/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 11:41:21,508 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 11:41:21,509 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:40909/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 11:41:21,734 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 11:41:21,734 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 11:41:21,734 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:40909/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 11:41:21.954 UTC [17344] WARNING: there is no transaction in progress
2026-09-26 11:41:21,959 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 11:41:21,960 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:40909/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 11:41:22,199 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 11:41:22,199 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 11:41:22,199 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:40909/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 11:41:22,419 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:40909/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 11:41:22,669 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:40909/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-xxxx.c.gcp-upstream.internal','N') ;"
2026-09-26 11:41:22,902 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:40909/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 11:41:23,156 [I] Table x_portal_user does not exist in database ranger
2026-09-26 11:41:23,156 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:40909/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 11:41:23,400 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 11:41:23,401 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:40909/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 11:41:23,630 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:40909/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 11:41:23,879 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 11:41:23,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:40909/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 11:41:24.781 UTC [17452] WARNING: there is no transaction in progress
2026-09-26 11:41:24.799 UTC [17452] WARNING: there is no transaction in progress
2026-09-26 11:41:24.816 UTC [17452] WARNING: there is no transaction in progress
2026-09-26 11:41:24.834 UTC [17452] WARNING: there is no transaction in progress
2026-09-26 11:41:24.852 UTC [17452] WARNING: there is no transaction in progress
2026-09-26 11:41:24.870 UTC [17452] WARNING: there is no transaction in progress
2026-09-26 11:41:24.926 UTC [17452] WARNING: there is no transaction in progress
2026-09-26 11:41:24.932 UTC [17452] WARNING: there is no transaction in progress
2026-09-26 11:41:24.945 UTC [17452] WARNING: there is no transaction in progress
2026-09-26 11:41:24.956 UTC [17452] WARNING: there is no transaction in progress
2026-09-26 11:41:24.965 UTC [17452] WARNING: there is no transaction in progress
2026-09-26 11:41:24.978 UTC [17452] WARNING: there is no transaction in progress
2026-09-26 11:41:24.987 UTC [17452] WARNING: there is no transaction in progress
2026-09-26 11:41:24.993 UTC [17452] WARNING: there is no transaction in progress
2026-09-26 11:41:24.999 UTC [17452] WARNING: there is no transaction in progress
2026-09-26 11:41:25.005 UTC [17452] WARNING: there is no transaction in progress
2026-09-26 11:41:25,584 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 11:41:25,584 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:40909/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 11:41:25,809 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 11:41:25,810 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:40909/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 11:41:26,031 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 11:41:26,031 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:40909/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 11:41:26,276 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 11:41:26,276 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:40909/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 11:41:26,516 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:40909/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 11:41:26,768 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:40909/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-xxxx.c.gcp-upstream.internal';"
2026-09-26 11:41:26,983 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 11:41:26,984 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:40909/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 11:41:27,207 [I] DB_PATCHES have already been applied
I20260926 11:41:27.212472 6528 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ranger-admin:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskBunNuW/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 11:41:27.212536 6528 mini_ranger.cc:193] Using host: 127.6.96.20
I20260926 11:41:27.215688 6528 mini_ranger.cc:227] Ranger admin URL: http://127.6.96.20:43983
Sep 26, 2026 11:41:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 11:41:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 11:41:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 11:41:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 11:41:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 11:41:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 11:41:28 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 11:41:28.227077 6528 mini_ranger.cc:160] Time spent starting Ranger: real 7.933s user 0.007s sys 0.002s
Sep 26, 2026 11:41:47 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 11:41:47 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 26, 2026 11:41:47 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 11:41:47 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 11:41:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 11:41:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 11:41:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 11:41:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 11:41:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 11:41:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 11:41:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 11:41:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 11:41:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 11:41:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 11:41:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 11:41:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 11:41:47 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 11:41:48 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 11:41:49.177731 6528 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.96.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.96.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.96.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/rangerkms_127.6.96.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.96.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/rangerkms_127.6.96.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.96.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.96.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.96.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/HTTP_127.6.96.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.96.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/HTTP_127.6.96.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 11:41:49 dist-test-slave-xxxx krb5kdc[17239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422909, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 11:41:49.259545 6528 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 11:41:49.268311 6528 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 11:41:50.078855 6528 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 11:41:50.078943 6528 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ranger-kms
I20260926 11:41:50.079033 6528 mini_ranger_kms.cc:227] Using postgres at 127.6.96.20:40909
2026-09-26 11:41:50,250 [I] DB FLAVOR :POSTGRES
2026-09-26 11:41:50,251 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 11:41:50,251 [I] Checking connection
2026-09-26 11:41:50,251 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:40909/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 11:41:50,475 [I] connection success
2026-09-26 11:41:50,476 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 11:41:50,476 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 11:41:50,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:40909/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 11:41:50,707 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 11:41:50,708 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 11:41:50,708 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 11:41:50,708 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:40909/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 11:41:50,955 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 11:41:51.373260 6528 mini_ranger_kms.cc:310] Created kms service
I20260926 11:41:51.544277 6528 mini_ranger_kms.cc:326] Created kudu user
I20260926 11:41:51.587997 6528 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 11:41:51 dist-test-slave-xxxx krb5kdc[17239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422909, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.96.20@KRBTEST.COM
I20260926 11:41:51.960484 6528 mini_ranger_kms.cc:384] Added ranger policy
I20260926 11:41:51.960639 6528 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ranger-kms:/tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 11:41:51.960664 6528 mini_ranger_kms.cc:240] Using host: 127.6.96.20
I20260926 11:41:51.964027 6528 mini_ranger_kms.cc:276] Ranger KMS PID: 17632
I20260926 11:41:51.964164 6528 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.96.20:38677
Sep 26, 2026 11:41:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 11:41:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 11:41:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 11:41:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 11:41:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 11:41:53 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 11:41:53.978614 6528 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.719s user 0.011s sys 0.145s
I20260926 11:41:53.978744 6528 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 11:41:53.978873 6528 mini_ranger_kms.cc:401] 127.6.96.20:38677/kms/v1/keys
Sep 26 11:41:54 dist-test-slave-xxxx krb5kdc[17239](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790422914, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.96.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 11:41:54 dist-test-slave-xxxx krb5kdc[17239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422914, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.96.20@KRBTEST.COM for HTTP/127.6.96.20@KRBTEST.COM
Sep 26 11:41:55 dist-test-slave-xxxx krb5kdc[17239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422914, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.96.20@KRBTEST.COM for HTTP/127.6.96.20@KRBTEST.COM
Sep 26 11:41:57 dist-test-slave-xxxx krb5kdc[17239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:41:57.249279 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:42733
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:36207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:42733
--ranger_config_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.96.20:38677/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:41:57.358162 17683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:41:57.358402 17683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:41:57.358438 17683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:41:57.362308 17683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:41:57.362363 17683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:41:57.362380 17683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:41:57.362394 17683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:41:57.362408 17683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:41:57.367641 17683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:36207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.96.20:38677/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:42733
--ranger_config_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:42733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:41:57.368541 17683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:41:57.369450 17683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:41:57.377594 17689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:41:57.377566 17691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:41:57.377568 17688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:57.377746 17683 server_base.cc:1034] running on GCE node
I20260926 11:41:57.378363 17683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:41:57.379460 17683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:41:57.380602 17683 hybrid_clock.cc:648] HybridClock initialized: now 1790422917380582 us; error 37 us; skew 500 ppm
Sep 26 11:41:57 dist-test-slave-xxxx krb5kdc[17239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422917, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:41:57.383123 17683 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:41:57.383966 17683 webserver.cc:460] Webserver started at http://127.6.96.62:42497/ using document root <none> and password file <none>
I20260926 11:41:57.384428 17683 fs_manager.cc:359] Metadata directory not provided
I20260926 11:41:57.384486 17683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:41:57.384641 17683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 11:41:57 dist-test-slave-xxxx krb5kdc[17239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422917, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for HTTP/127.6.96.20@KRBTEST.COM
I20260926 11:41:57.439399 17683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "d0ca3bfd4f1c4a549c666d5a7c68eaea"
format_stamp: "Formatted at 2026-09-26 11:41:57 on dist-test-slave-xxxx"
server_key: "223dca2e400aaf63b15c2030c257b664"
server_key_iv: "4c563a186e3ccd836ef0a8ee4c4ea3a4"
server_key_version: "kuduclusterkey@0"
I20260926 11:41:57.440164 17683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "d0ca3bfd4f1c4a549c666d5a7c68eaea"
format_stamp: "Formatted at 2026-09-26 11:41:57 on dist-test-slave-xxxx"
server_key: "223dca2e400aaf63b15c2030c257b664"
server_key_iv: "4c563a186e3ccd836ef0a8ee4c4ea3a4"
server_key_version: "kuduclusterkey@0"
I20260926 11:41:57.443840 17683 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 11:41:57.461164 17698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:57.462580 17683 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 11:41:57.462703 17683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "d0ca3bfd4f1c4a549c666d5a7c68eaea"
format_stamp: "Formatted at 2026-09-26 11:41:57 on dist-test-slave-xxxx"
server_key: "223dca2e400aaf63b15c2030c257b664"
server_key_iv: "4c563a186e3ccd836ef0a8ee4c4ea3a4"
server_key_version: "kuduclusterkey@0"
I20260926 11:41:57.462817 17683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:41:57.508632 17683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:41:57.511512 17683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:41:57.511694 17683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:41:57.519104 17683 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:42733
I20260926 11:41:57.519129 17750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:42733 every 8 connection(s)
I20260926 11:41:57.520103 17683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:41:57.522945 17751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:41:57.526783 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 17683
I20260926 11:41:57.526958 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:41:57.527243 6528 external_mini_cluster.cc:1428] Setting key 0817e0046a2085499b760a1ae87d9c4e
I20260926 11:41:57.528576 17751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0ca3bfd4f1c4a549c666d5a7c68eaea: Bootstrap starting.
I20260926 11:41:57.531235 17751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0ca3bfd4f1c4a549c666d5a7c68eaea: Neither blocks nor log segments found. Creating new log.
I20260926 11:41:57.532038 17751 log.cc:824] T 00000000000000000000000000000000 P d0ca3bfd4f1c4a549c666d5a7c68eaea: Log is configured to *not* fsync() on all Append() calls
I20260926 11:41:57.533972 17751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0ca3bfd4f1c4a549c666d5a7c68eaea: No bootstrap required, opened a new log
Sep 26 11:41:57 dist-test-slave-xxxx krb5kdc[17239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:41:57.537947 17751 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0ca3bfd4f1c4a549c666d5a7c68eaea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0ca3bfd4f1c4a549c666d5a7c68eaea" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 42733 } }
I20260926 11:41:57.538167 17751 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0ca3bfd4f1c4a549c666d5a7c68eaea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:41:57.538234 17751 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0ca3bfd4f1c4a549c666d5a7c68eaea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0ca3bfd4f1c4a549c666d5a7c68eaea, State: Initialized, Role: FOLLOWER
I20260926 11:41:57.538712 17751 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0ca3bfd4f1c4a549c666d5a7c68eaea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0ca3bfd4f1c4a549c666d5a7c68eaea" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 42733 } }
I20260926 11:41:57.538867 17751 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0ca3bfd4f1c4a549c666d5a7c68eaea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:41:57.538925 17751 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0ca3bfd4f1c4a549c666d5a7c68eaea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:41:57.539058 17751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0ca3bfd4f1c4a549c666d5a7c68eaea [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:41:57.539897 17751 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0ca3bfd4f1c4a549c666d5a7c68eaea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0ca3bfd4f1c4a549c666d5a7c68eaea" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 42733 } }
I20260926 11:41:57.540261 17751 leader_election.cc:302] T 00000000000000000000000000000000 P d0ca3bfd4f1c4a549c666d5a7c68eaea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0ca3bfd4f1c4a549c666d5a7c68eaea; no voters:
I20260926 11:41:57.540685 17751 leader_election.cc:288] T 00000000000000000000000000000000 P d0ca3bfd4f1c4a549c666d5a7c68eaea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:41:57.540717 17756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0ca3bfd4f1c4a549c666d5a7c68eaea [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:41:57.541062 17756 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0ca3bfd4f1c4a549c666d5a7c68eaea [term 1 LEADER]: Becoming Leader. State: Replica: d0ca3bfd4f1c4a549c666d5a7c68eaea, State: Running, Role: LEADER
I20260926 11:41:57.541432 17756 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0ca3bfd4f1c4a549c666d5a7c68eaea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0ca3bfd4f1c4a549c666d5a7c68eaea" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 42733 } }
I20260926 11:41:57.542088 17751 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0ca3bfd4f1c4a549c666d5a7c68eaea [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:41:57.542603 17756 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0ca3bfd4f1c4a549c666d5a7c68eaea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0ca3bfd4f1c4a549c666d5a7c68eaea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0ca3bfd4f1c4a549c666d5a7c68eaea" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 42733 } } }
I20260926 11:41:57.542789 17756 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0ca3bfd4f1c4a549c666d5a7c68eaea [sys.catalog]: This master's current role is: LEADER
I20260926 11:41:57.543087 17756 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0ca3bfd4f1c4a549c666d5a7c68eaea [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0ca3bfd4f1c4a549c666d5a7c68eaea. Latest consensus state: current_term: 1 leader_uuid: "d0ca3bfd4f1c4a549c666d5a7c68eaea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0ca3bfd4f1c4a549c666d5a7c68eaea" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 42733 } } }
I20260926 11:41:57.543249 17756 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0ca3bfd4f1c4a549c666d5a7c68eaea [sys.catalog]: This master's current role is: LEADER
I20260926 11:41:57.543545 17763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:41:57.543767 17754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:57.528709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59976 (local address 127.6.96.62:42733)
0926 11:41:57.529153 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:57.529171 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:57.529210 (+ 39us) server_negotiation.cc:407] Connection header received
0926 11:41:57.530055 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:57.530077 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:57.530483 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:57.530859 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:57.531709 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:57.532977 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:57.534177 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:57.534867 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:57.535019 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:57.537927 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:41:57.537933 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:41:57.537951 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:41:57.537982 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:41:57.539948 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:41:57.540520 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:41:57.540527 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:41:57.540533 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:41:57.540665 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:41:57.540981 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:41:57.540985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:41:57.540987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:41:57.541475 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:41:57.542264 (+ 789us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:57.542409 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 11:41:57.542699 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":141,"threads_started":1}
I20260926 11:41:57.547014 17763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:41:57.547367 17751 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 11:41:57.561201 17763 catalog_manager.cc:1347] Generated new cluster ID: ea002a20c05d4a039de42c3a3a2e3a86
I20260926 11:41:57.561370 17763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:41:57.601836 17763 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:41:57.603185 17763 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:41:57.609043 17763 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0ca3bfd4f1c4a549c666d5a7c68eaea: Generated new TSK 0
I20260926 11:41:57.609822 17763 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.96.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
I20260926 11:41:58.262316 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.96.20:38677/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:42733
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:36207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:41:58.402261 17796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:41:58.402525 17796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:41:58.402583 17796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:41:58.407835 17796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:41:58.407912 17796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:41:58.407998 17796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:41:58.415666 17796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:36207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.96.20:38677/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:42733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:41:58.417091 17796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:41:58.418310 17796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:41:58.427996 17806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:58.428141 17796 server_base.cc:1034] running on GCE node
W20260926 11:41:58.427999 17803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:41:58.430959 17804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:58.431362 17796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:41:58.431990 17796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:41:58.433144 17796 hybrid_clock.cc:648] HybridClock initialized: now 1790422918433120 us; error 40 us; skew 500 ppm
Sep 26 11:41:58 dist-test-slave-xxxx krb5kdc[17239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422918, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:41:58.436000 17796 init.cc:377] Logged in from keytab as kudu/127.6.96.1@KRBTEST.COM (short username kudu)
I20260926 11:41:58.436892 17796 webserver.cc:460] Webserver started at http://127.6.96.1:38809/ using document root <none> and password file <none>
I20260926 11:41:58.437623 17796 fs_manager.cc:359] Metadata directory not provided
I20260926 11:41:58.437757 17796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:41:58.438028 17796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 11:41:58 dist-test-slave-xxxx krb5kdc[17239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422918, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for HTTP/127.6.96.20@KRBTEST.COM
I20260926 11:41:58.460940 17796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "88bd63e4177244249ea720769108df1c"
format_stamp: "Formatted at 2026-09-26 11:41:58 on dist-test-slave-xxxx"
server_key: "aab06cce5f0564513fd5e16a0fca6c93"
server_key_iv: "9c6a08bddbeac506729006ef38c5eb74"
server_key_version: "kuduclusterkey@0"
I20260926 11:41:58.461789 17796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "88bd63e4177244249ea720769108df1c"
format_stamp: "Formatted at 2026-09-26 11:41:58 on dist-test-slave-xxxx"
server_key: "aab06cce5f0564513fd5e16a0fca6c93"
server_key_iv: "9c6a08bddbeac506729006ef38c5eb74"
server_key_version: "kuduclusterkey@0"
I20260926 11:41:58.467072 17796 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 11:41:58.483253 17815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:58.484884 17796 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 11:41:58.485016 17796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "88bd63e4177244249ea720769108df1c"
format_stamp: "Formatted at 2026-09-26 11:41:58 on dist-test-slave-xxxx"
server_key: "aab06cce5f0564513fd5e16a0fca6c93"
server_key_iv: "9c6a08bddbeac506729006ef38c5eb74"
server_key_version: "kuduclusterkey@0"
I20260926 11:41:58.485142 17796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:41:58.504841 17796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:41:58.508531 17796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:41:58.508719 17796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:41:58.509383 17796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:41:58.510385 17796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:41:58.510442 17796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:58.510495 17796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:41:58.510524 17796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:58.521870 17796 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:42203
I20260926 11:41:58.523464 17928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:42203 every 8 connection(s)
I20260926 11:41:58.524151 17796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:41:58.532341 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 17796
I20260926 11:41:58.532470 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:41:58.532735 6528 external_mini_cluster.cc:1428] Setting key 809a46e4752f4e7b15ffcb4025e046b9
Sep 26 11:41:58 dist-test-slave-xxxx krb5kdc[17239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422918, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:41:58.542807 17932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:58.528343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:39391 (local address 127.6.96.62:42733)
0926 11:41:58.528626 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:58.528635 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:58.529048 (+ 413us) server_negotiation.cc:407] Connection header received
0926 11:41:58.530236 (+ 1188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:58.530240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:58.530295 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:58.530353 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:58.531987 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:58.532846 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:58.534950 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:58.535291 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:58.535369 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:58.538573 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:41:58.538577 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:41:58.538580 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:41:58.538606 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:41:58.540416 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:41:58.541029 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:41:58.541033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:41:58.541035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:41:58.541123 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:41:58.541510 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:41:58.541513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:41:58.541515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:41:58.541732 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:41:58.541807 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:58.542501 (+ 694us) server_negotiation.cc:299] Negotiation successful
0926 11:41:58.542630 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for kudu/127.6.96.2@KRBTEST.COM; defaulting to no policy
I20260926 11:41:58.545671 17931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:58.527330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:42733 (local address 127.6.96.1:39391)
0926 11:41:58.528889 (+ 1559us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:58.528941 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:58.529999 (+ 1058us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:58.530740 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:58.530749 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:58.531159 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:58.531803 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:58.531821 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:58.533008 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:58.533012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:58.534781 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:58.534792 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:58.535573 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:58.535577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:58.535774 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:58.536425 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:58.536445 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:58.538381 (+ 1936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:41:58.540543 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:41:58.540549 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:41:58.540565 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:41:58.540911 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:41:58.541231 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:41:58.541234 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:41:58.541237 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:41:58.541408 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:41:58.541886 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:41:58.541892 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:41:58.542221 (+ 329us) client_negotiation.cc:769] Sending connection context
0926 11:41:58.543497 (+ 1276us) client_negotiation.cc:240] Negotiation successful
0926 11:41:58.544071 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1353,"thread_start_us":148,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.2@KRBTEST.COM" created.
I20260926 11:41:58.547063 17929 heartbeater.cc:347] Connected to a master server at 127.6.96.62:42733
I20260926 11:41:58.547319 17929 heartbeater.cc:464] Registering TS with master...
I20260926 11:41:58.547819 17929 heartbeater.cc:511] Master 127.6.96.62:42733 requested a full tablet report, sending...
I20260926 11:41:58.549508 17715 ts_manager.cc:194] Registered new tserver with Master: 88bd63e4177244249ea720769108df1c (127.6.96.1:42203)
I20260926 11:41:58.551126 17715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.1@KRBTEST.COM'} at 127.6.96.1:39391
Sep 26 11:41:58 dist-test-slave-xxxx krb5kdc[17239](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790422918, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
I20260926 11:41:58.562783 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.96.20:38677/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:42733
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:36207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:41:58.722994 17935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:41:58.723345 17935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:41:58.723421 17935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:41:58.729274 17935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:41:58.729352 17935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:41:58.729453 17935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:41:58.737129 17935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:36207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.96.20:38677/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:42733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:41:58.738432 17935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:41:58.739766 17935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:41:58.752061 17944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:41:58.752063 17942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:58.752885 17935 server_base.cc:1034] running on GCE node
W20260926 11:41:58.752923 17941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:58.753479 17935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:41:58.754344 17935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:41:58.755546 17935 hybrid_clock.cc:648] HybridClock initialized: now 1790422918755522 us; error 38 us; skew 500 ppm
Sep 26 11:41:58 dist-test-slave-xxxx krb5kdc[17239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422918, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:41:58.759001 17935 init.cc:377] Logged in from keytab as kudu/127.6.96.2@KRBTEST.COM (short username kudu)
I20260926 11:41:58.760008 17935 webserver.cc:460] Webserver started at http://127.6.96.2:36661/ using document root <none> and password file <none>
I20260926 11:41:58.760746 17935 fs_manager.cc:359] Metadata directory not provided
I20260926 11:41:58.760810 17935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:41:58.761024 17935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 11:41:58 dist-test-slave-xxxx krb5kdc[17239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422918, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for HTTP/127.6.96.20@KRBTEST.COM
I20260926 11:41:58.779877 17935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "0f6a5181973f4384ab5fb77475523792"
format_stamp: "Formatted at 2026-09-26 11:41:58 on dist-test-slave-xxxx"
server_key: "b25d2d7250de72f448c0c01ef277c42d"
server_key_iv: "cceea7cca5d0c6aa1d88d3476784de39"
server_key_version: "kuduclusterkey@0"
I20260926 11:41:58.780674 17935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "0f6a5181973f4384ab5fb77475523792"
format_stamp: "Formatted at 2026-09-26 11:41:58 on dist-test-slave-xxxx"
server_key: "b25d2d7250de72f448c0c01ef277c42d"
server_key_iv: "cceea7cca5d0c6aa1d88d3476784de39"
server_key_version: "kuduclusterkey@0"
I20260926 11:41:58.795737 17935 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.004s sys 0.000s
I20260926 11:41:58.814581 17951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:58.816306 17935 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 11:41:58.816449 17935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "0f6a5181973f4384ab5fb77475523792"
format_stamp: "Formatted at 2026-09-26 11:41:58 on dist-test-slave-xxxx"
server_key: "b25d2d7250de72f448c0c01ef277c42d"
server_key_iv: "cceea7cca5d0c6aa1d88d3476784de39"
server_key_version: "kuduclusterkey@0"
I20260926 11:41:58.816586 17935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:41:58.840536 17935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:41:58.843569 17935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:41:58.843730 17935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:41:58.844327 17935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:41:58.845206 17935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:41:58.845252 17935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:58.845302 17935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:41:58.845328 17935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:58.854938 17935 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:45757
I20260926 11:41:58.855918 17935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:41:58.860106 18064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:45757 every 8 connection(s)
I20260926 11:41:58.863682 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 17935
I20260926 11:41:58.863803 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:41:58.864071 6528 external_mini_cluster.cc:1428] Setting key 987707587af458de62eaea34d85dee07
Sep 26 11:41:58 dist-test-slave-xxxx krb5kdc[17239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422918, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.96.3@KRBTEST.COM; defaulting to no policy
I20260926 11:41:58.888571 17932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:58.864851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:51011 (local address 127.6.96.62:42733)
0926 11:41:58.865165 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:58.865171 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:58.875135 (+ 9964us) server_negotiation.cc:407] Connection header received
0926 11:41:58.876441 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:58.876445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:58.876498 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:58.876548 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:58.878282 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:58.879129 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:58.880873 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:58.881217 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:58.881289 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:58.884205 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:41:58.884209 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:41:58.884213 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:41:58.884249 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:41:58.885861 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:41:58.886455 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:41:58.886459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:41:58.886461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:41:58.886545 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:41:58.887001 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:41:58.887004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:41:58.887006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:41:58.887231 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:41:58.887305 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:58.888236 (+ 931us) server_negotiation.cc:299] Negotiation successful
0926 11:41:58.888383 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.3@KRBTEST.COM" created.
I20260926 11:41:58.889772 18067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:58.862472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:42733 (local address 127.6.96.2:51011)
0926 11:41:58.874948 (+ 12476us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:58.875021 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:58.876170 (+ 1149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:58.876969 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:58.876978 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:58.877329 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:58.878115 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:58.878136 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:58.879334 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:58.879338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:58.880742 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:58.880752 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:58.881497 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:58.881501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:58.881689 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:58.882282 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:58.882301 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:58.884014 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:41:58.885985 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:41:58.885990 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:41:58.886008 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:41:58.886330 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:41:58.886647 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:41:58.886650 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:41:58.886652 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:41:58.886896 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:41:58.887396 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:41:58.887403 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:41:58.887733 (+ 330us) client_negotiation.cc:769] Sending connection context
0926 11:41:58.888122 (+ 389us) client_negotiation.cc:240] Negotiation successful
0926 11:41:58.888477 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12175,"thread_start_us":139,"threads_started":1}
I20260926 11:41:58.891131 18065 heartbeater.cc:347] Connected to a master server at 127.6.96.62:42733
I20260926 11:41:58.891386 18065 heartbeater.cc:464] Registering TS with master...
I20260926 11:41:58.891937 18065 heartbeater.cc:511] Master 127.6.96.62:42733 requested a full tablet report, sending...
I20260926 11:41:58.893107 17715 ts_manager.cc:194] Registered new tserver with Master: 0f6a5181973f4384ab5fb77475523792 (127.6.96.2:45757)
I20260926 11:41:58.894001 17715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.2@KRBTEST.COM'} at 127.6.96.2:51011
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
I20260926 11:41:58.907517 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.96.20:38677/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:42733
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:36207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 11:41:59 dist-test-slave-xxxx krb5kdc[17239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422918, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for HTTP/127.6.96.20@KRBTEST.COM
W20260926 11:41:59.064978 18070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:41:59.065245 18070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:41:59.065303 18070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:41:59.069478 18070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:41:59.069545 18070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:41:59.069628 18070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:41:59.076825 18070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:36207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.96.20:38677/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:42733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:41:59.078404 18070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:41:59.081022 18070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:41:59.096518 18078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:41:59.097250 18075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:59.098189 18070 server_base.cc:1034] running on GCE node
W20260926 11:41:59.098409 18076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:41:59.098942 18070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:41:59.099754 18070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:41:59.107127 18070 hybrid_clock.cc:648] HybridClock initialized: now 1790422919107075 us; error 62 us; skew 500 ppm
Sep 26 11:41:59 dist-test-slave-xxxx krb5kdc[17239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422919, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:41:59.110780 18070 init.cc:377] Logged in from keytab as kudu/127.6.96.3@KRBTEST.COM (short username kudu)
I20260926 11:41:59.112003 18070 webserver.cc:460] Webserver started at http://127.6.96.3:40621/ using document root <none> and password file <none>
I20260926 11:41:59.113193 18070 fs_manager.cc:359] Metadata directory not provided
I20260926 11:41:59.113338 18070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:41:59.113631 18070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 11:41:59 dist-test-slave-xxxx krb5kdc[17239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422919, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for HTTP/127.6.96.20@KRBTEST.COM
I20260926 11:41:59.149472 18070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "4ffbb7291796495b9b2a888aa73eec9f"
format_stamp: "Formatted at 2026-09-26 11:41:59 on dist-test-slave-xxxx"
server_key: "21b1e8411f8ebc9a0a590614dde8f48b"
server_key_iv: "ebee292822f2cffaf4dda2eee8b6deda"
server_key_version: "kuduclusterkey@0"
I20260926 11:41:59.156038 18070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "4ffbb7291796495b9b2a888aa73eec9f"
format_stamp: "Formatted at 2026-09-26 11:41:59 on dist-test-slave-xxxx"
server_key: "21b1e8411f8ebc9a0a590614dde8f48b"
server_key_iv: "ebee292822f2cffaf4dda2eee8b6deda"
server_key_version: "kuduclusterkey@0"
I20260926 11:41:59.165253 18070 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.000s sys 0.004s
I20260926 11:41:59.196007 18085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:59.197996 18070 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.001s
I20260926 11:41:59.198146 18070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "4ffbb7291796495b9b2a888aa73eec9f"
format_stamp: "Formatted at 2026-09-26 11:41:59 on dist-test-slave-xxxx"
server_key: "21b1e8411f8ebc9a0a590614dde8f48b"
server_key_iv: "ebee292822f2cffaf4dda2eee8b6deda"
server_key_version: "kuduclusterkey@0"
I20260926 11:41:59.198302 18070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:41:59.212514 18070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:41:59.216125 18070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:41:59.216382 18070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:41:59.217228 18070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:41:59.218336 18070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:41:59.218456 18070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:59.218562 18070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:41:59.218639 18070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:41:59.229883 18070 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:44065
I20260926 11:41:59.234215 18070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:41:59.237183 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 18070
I20260926 11:41:59.237303 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:41:59.237584 6528 external_mini_cluster.cc:1428] Setting key 0b9bc26b35a496b020732c3ef7c2dea1
I20260926 11:41:59.235953 18198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:44065 every 8 connection(s)
Sep 26 11:41:59 dist-test-slave-xxxx krb5kdc[17239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422919, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:41:59.265750 17932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:59.239694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:47191 (local address 127.6.96.62:42733)
0926 11:41:59.239824 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:59.239828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:59.247368 (+ 7540us) server_negotiation.cc:407] Connection header received
0926 11:41:59.253073 (+ 5705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:59.253077 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:59.253130 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:59.253191 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:59.254563 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.255391 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.257223 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.257558 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.257625 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:59.261911 (+ 4286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:41:59.261915 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:41:59.261918 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:41:59.261944 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:41:59.263527 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:41:59.264091 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:41:59.264094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:41:59.264096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:41:59.264175 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:41:59.264540 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:41:59.264543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:41:59.264545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:41:59.264740 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:41:59.264804 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:59.265468 (+ 664us) server_negotiation.cc:299] Negotiation successful
0926 11:41:59.265589 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 11:41:59.267400 18201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:59.240084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:42733 (local address 127.6.96.3:47191)
0926 11:41:59.247189 (+ 7105us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:59.247257 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:59.252827 (+ 5570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:59.253382 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:59.253389 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:59.253669 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:41:59.254391 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:59.254412 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.255704 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.255708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:59.257094 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:59.257105 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.259356 (+ 2251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.259359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:59.259557 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:59.260153 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:41:59.260173 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:41:59.261732 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:41:59.263639 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:41:59.263645 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:41:59.263662 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:41:59.263986 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:41:59.264281 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:41:59.264284 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:41:59.264286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:41:59.264446 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:41:59.264876 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:41:59.264882 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:41:59.265192 (+ 310us) client_negotiation.cc:769] Sending connection context
0926 11:41:59.265947 (+ 755us) client_negotiation.cc:240] Negotiation successful
0926 11:41:59.266295 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6833,"thread_start_us":140,"threads_started":1}
I20260926 11:41:59.268478 18199 heartbeater.cc:347] Connected to a master server at 127.6.96.62:42733
I20260926 11:41:59.268716 18199 heartbeater.cc:464] Registering TS with master...
I20260926 11:41:59.269236 18199 heartbeater.cc:511] Master 127.6.96.62:42733 requested a full tablet report, sending...
I20260926 11:41:59.270222 17715 ts_manager.cc:194] Registered new tserver with Master: 4ffbb7291796495b9b2a888aa73eec9f (127.6.96.3:44065)
I20260926 11:41:59.270972 17715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.3@KRBTEST.COM'} at 127.6.96.3:47191
I20260926 11:41:59.275386 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 11:41:59.292430 17932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:59.279240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59982 (local address 127.6.96.62:42733)
0926 11:41:59.279365 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:59.279369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:59.283108 (+ 3739us) server_negotiation.cc:407] Connection header received
0926 11:41:59.283298 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:59.283302 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:59.283348 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:59.283397 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:41:59.284303 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.285074 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.286238 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.286579 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.286650 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:59.288415 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:41:59.288419 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:41:59.288422 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:41:59.288449 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:41:59.290557 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:41:59.291106 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:41:59.291110 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:41:59.291112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:41:59.291194 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:41:59.291496 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:41:59.291499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:41:59.291501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:41:59.291706 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:41:59.291811 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:59.292140 (+ 329us) server_negotiation.cc:299] Negotiation successful
0926 11:41:59.292271 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 11:41:59.296308 17715 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 11:41:59.300640 17715 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:41:59.327163 18217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:59.316354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41362 (local address 127.6.96.1:42203)
0926 11:41:59.317744 (+ 1390us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:59.317749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:59.318669 (+ 920us) server_negotiation.cc:407] Connection header received
0926 11:41:59.318749 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:59.318754 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:59.319003 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:59.319133 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:41:59.319569 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.321389 (+ 1820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.323165 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.325995 (+ 2830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.326083 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:59.326239 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:59.326915 (+ 676us) server_negotiation.cc:299] Negotiation successful
0926 11:41:59.327024 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1301,"thread_start_us":72,"threads_started":1}
I20260926 11:41:59.327947 18216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:59.313100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:44065 (local address 127.0.0.1:48394)
0926 11:41:59.317923 (+ 4823us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:59.317975 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:59.318101 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:59.322951 (+ 4850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:59.322954 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:59.322972 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:41:59.323052 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:59.323057 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.324598 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.324601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:59.325752 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:59.325761 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.327577 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.327580 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:59.327668 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:59.327692 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 11:41:59.327758 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 11:41:59.327821 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3800,"thread_start_us":100,"threads_started":1}
I20260926 11:41:59.328624 18213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:59.314021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48394 (local address 127.6.96.3:44065)
0926 11:41:59.317663 (+ 3642us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:59.317676 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:59.321792 (+ 4116us) server_negotiation.cc:407] Connection header received
0926 11:41:59.321911 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:59.321917 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:59.322184 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:59.322335 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:41:59.323261 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.324300 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.326074 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.326880 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.326972 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:59.327104 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:59.328391 (+ 1287us) server_negotiation.cc:299] Negotiation successful
0926 11:41:59.328490 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3524,"thread_start_us":118,"threads_started":1}
I20260926 11:41:59.329340 18214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:59.316014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:42203 (local address 127.0.0.1:41362)
0926 11:41:59.318521 (+ 2507us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:59.318530 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:59.318590 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:59.319302 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:59.319308 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:59.319330 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:41:59.319452 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:59.319458 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.321513 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.321516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:59.322807 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:59.322817 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.326320 (+ 3503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.326323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:59.326435 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:59.326473 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 11:41:59.326574 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 11:41:59.326693 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2463,"thread_start_us":67,"threads_started":1}
I20260926 11:41:59.331833 18132 tablet_service.cc:1467] Processing CreateTablet for tablet 36f637b3381145f88b83395c5561a4e0 (DEFAULT_TABLE table=test-table [id=c0fae4c5ad024619a10fc34847bdc6e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:41:59.332170 17859 tablet_service.cc:1467] Processing CreateTablet for tablet 36f637b3381145f88b83395c5561a4e0 (DEFAULT_TABLE table=test-table [id=c0fae4c5ad024619a10fc34847bdc6e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:41:59.333024 18132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36f637b3381145f88b83395c5561a4e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:41:59.333024 17859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36f637b3381145f88b83395c5561a4e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:41:59.342670 18219 tablet_bootstrap.cc:492] T 36f637b3381145f88b83395c5561a4e0 P 88bd63e4177244249ea720769108df1c: Bootstrap starting.
I20260926 11:41:59.345625 18219 tablet_bootstrap.cc:654] T 36f637b3381145f88b83395c5561a4e0 P 88bd63e4177244249ea720769108df1c: Neither blocks nor log segments found. Creating new log.
I20260926 11:41:59.346541 18219 log.cc:824] T 36f637b3381145f88b83395c5561a4e0 P 88bd63e4177244249ea720769108df1c: Log is configured to *not* fsync() on all Append() calls
I20260926 11:41:59.348724 18219 tablet_bootstrap.cc:492] T 36f637b3381145f88b83395c5561a4e0 P 88bd63e4177244249ea720769108df1c: No bootstrap required, opened a new log
I20260926 11:41:59.348942 18219 ts_tablet_manager.cc:1397] T 36f637b3381145f88b83395c5561a4e0 P 88bd63e4177244249ea720769108df1c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 11:41:59.349673 18212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:59.315425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:45757 (local address 127.0.0.1:39274)
0926 11:41:59.318313 (+ 2888us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:59.318324 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:59.318387 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:59.324630 (+ 6243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:59.324633 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:59.324649 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:41:59.324731 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:59.324736 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.327155 (+ 2419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.327159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:59.328246 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:59.328256 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.349182 (+ 20926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.349187 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:59.349307 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:59.349346 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 11:41:59.349430 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 11:41:59.349522 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2833,"thread_start_us":85,"threads_started":1}
I20260926 11:41:59.352113 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:59.315552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39274 (local address 127.6.96.2:45757)
0926 11:41:59.322666 (+ 7114us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:59.322681 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:59.322702 (+ 21us) server_negotiation.cc:407] Connection header received
0926 11:41:59.322783 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:59.322788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:59.323096 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:59.324499 (+ 1403us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:41:59.324862 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.325940 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.347905 (+ 21965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.348796 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.348927 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:59.349087 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:59.351856 (+ 2769us) server_negotiation.cc:299] Negotiation successful
0926 11:41:59.351969 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7006,"thread_start_us":92,"threads_started":1}
I20260926 11:41:59.353691 18219 raft_consensus.cc:348] T 36f637b3381145f88b83395c5561a4e0 P 88bd63e4177244249ea720769108df1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffbb7291796495b9b2a888aa73eec9f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 44065 } } peers { permanent_uuid: "0f6a5181973f4384ab5fb77475523792" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 45757 } } peers { permanent_uuid: "88bd63e4177244249ea720769108df1c" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42203 } }
I20260926 11:41:59.353963 18219 raft_consensus.cc:374] T 36f637b3381145f88b83395c5561a4e0 P 88bd63e4177244249ea720769108df1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:41:59.354015 18219 raft_consensus.cc:729] T 36f637b3381145f88b83395c5561a4e0 P 88bd63e4177244249ea720769108df1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88bd63e4177244249ea720769108df1c, State: Initialized, Role: FOLLOWER
I20260926 11:41:59.354498 18219 consensus_queue.cc:260] T 36f637b3381145f88b83395c5561a4e0 P 88bd63e4177244249ea720769108df1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffbb7291796495b9b2a888aa73eec9f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 44065 } } peers { permanent_uuid: "0f6a5181973f4384ab5fb77475523792" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 45757 } } peers { permanent_uuid: "88bd63e4177244249ea720769108df1c" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42203 } }
I20260926 11:41:59.355541 18219 ts_tablet_manager.cc:1428] T 36f637b3381145f88b83395c5561a4e0 P 88bd63e4177244249ea720769108df1c: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260926 11:41:59.355130 17987 tablet_service.cc:1467] Processing CreateTablet for tablet 36f637b3381145f88b83395c5561a4e0 (DEFAULT_TABLE table=test-table [id=c0fae4c5ad024619a10fc34847bdc6e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:41:59.355881 17929 heartbeater.cc:503] Master 127.6.96.62:42733 was elected leader, sending a full tablet report...
I20260926 11:41:59.356206 17987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36f637b3381145f88b83395c5561a4e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:41:59.361941 18221 tablet_bootstrap.cc:492] T 36f637b3381145f88b83395c5561a4e0 P 4ffbb7291796495b9b2a888aa73eec9f: Bootstrap starting.
I20260926 11:41:59.364790 18221 tablet_bootstrap.cc:654] T 36f637b3381145f88b83395c5561a4e0 P 4ffbb7291796495b9b2a888aa73eec9f: Neither blocks nor log segments found. Creating new log.
I20260926 11:41:59.365559 18221 log.cc:824] T 36f637b3381145f88b83395c5561a4e0 P 4ffbb7291796495b9b2a888aa73eec9f: Log is configured to *not* fsync() on all Append() calls
I20260926 11:41:59.370421 18221 tablet_bootstrap.cc:492] T 36f637b3381145f88b83395c5561a4e0 P 4ffbb7291796495b9b2a888aa73eec9f: No bootstrap required, opened a new log
I20260926 11:41:59.370671 18221 ts_tablet_manager.cc:1397] T 36f637b3381145f88b83395c5561a4e0 P 4ffbb7291796495b9b2a888aa73eec9f: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260926 11:41:59.373824 18223 tablet_bootstrap.cc:492] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792: Bootstrap starting.
Sep 26 11:41:59 dist-test-slave-xxxx krb5kdc[17239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422918, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for HTTP/127.6.96.20@KRBTEST.COM
I20260926 11:41:59.376308 18223 tablet_bootstrap.cc:654] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792: Neither blocks nor log segments found. Creating new log.
I20260926 11:41:59.377609 18223 log.cc:824] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792: Log is configured to *not* fsync() on all Append() calls
I20260926 11:41:59.381973 18221 raft_consensus.cc:348] T 36f637b3381145f88b83395c5561a4e0 P 4ffbb7291796495b9b2a888aa73eec9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffbb7291796495b9b2a888aa73eec9f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 44065 } } peers { permanent_uuid: "0f6a5181973f4384ab5fb77475523792" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 45757 } } peers { permanent_uuid: "88bd63e4177244249ea720769108df1c" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42203 } }
I20260926 11:41:59.382280 18221 raft_consensus.cc:374] T 36f637b3381145f88b83395c5561a4e0 P 4ffbb7291796495b9b2a888aa73eec9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:41:59.382334 18221 raft_consensus.cc:729] T 36f637b3381145f88b83395c5561a4e0 P 4ffbb7291796495b9b2a888aa73eec9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ffbb7291796495b9b2a888aa73eec9f, State: Initialized, Role: FOLLOWER
I20260926 11:41:59.382903 18221 consensus_queue.cc:260] T 36f637b3381145f88b83395c5561a4e0 P 4ffbb7291796495b9b2a888aa73eec9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffbb7291796495b9b2a888aa73eec9f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 44065 } } peers { permanent_uuid: "0f6a5181973f4384ab5fb77475523792" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 45757 } } peers { permanent_uuid: "88bd63e4177244249ea720769108df1c" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42203 } }
I20260926 11:41:59.384027 18221 ts_tablet_manager.cc:1428] T 36f637b3381145f88b83395c5561a4e0 P 4ffbb7291796495b9b2a888aa73eec9f: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20260926 11:41:59.388782 18223 tablet_bootstrap.cc:492] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792: No bootstrap required, opened a new log
I20260926 11:41:59.389007 18223 ts_tablet_manager.cc:1397] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.004s
I20260926 11:41:59.393653 18223 raft_consensus.cc:348] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffbb7291796495b9b2a888aa73eec9f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 44065 } } peers { permanent_uuid: "0f6a5181973f4384ab5fb77475523792" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 45757 } } peers { permanent_uuid: "88bd63e4177244249ea720769108df1c" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42203 } }
I20260926 11:41:59.394228 18223 raft_consensus.cc:374] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:41:59.394291 18223 raft_consensus.cc:729] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f6a5181973f4384ab5fb77475523792, State: Initialized, Role: FOLLOWER
I20260926 11:41:59.395025 18199 heartbeater.cc:503] Master 127.6.96.62:42733 was elected leader, sending a full tablet report...
I20260926 11:41:59.395016 18223 consensus_queue.cc:260] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffbb7291796495b9b2a888aa73eec9f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 44065 } } peers { permanent_uuid: "0f6a5181973f4384ab5fb77475523792" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 45757 } } peers { permanent_uuid: "88bd63e4177244249ea720769108df1c" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42203 } }
I20260926 11:41:59.396361 18223 ts_tablet_manager.cc:1428] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260926 11:41:59.396688 18065 heartbeater.cc:503] Master 127.6.96.62:42733 was elected leader, sending a full tablet report...
W20260926 11:41:59.488751 18200 tablet.cc:2369] T 36f637b3381145f88b83395c5561a4e0 P 4ffbb7291796495b9b2a888aa73eec9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 11:41:59.527647 17930 tablet.cc:2369] T 36f637b3381145f88b83395c5561a4e0 P 88bd63e4177244249ea720769108df1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 11:41:59.616113 18066 tablet.cc:2369] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:41:59.655094 18227 raft_consensus.cc:482] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:41:59.655367 18227 raft_consensus.cc:504] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffbb7291796495b9b2a888aa73eec9f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 44065 } } peers { permanent_uuid: "0f6a5181973f4384ab5fb77475523792" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 45757 } } peers { permanent_uuid: "88bd63e4177244249ea720769108df1c" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42203 } }
I20260926 11:41:59.656836 18227 leader_election.cc:288] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ffbb7291796495b9b2a888aa73eec9f (127.6.96.3:44065), 88bd63e4177244249ea720769108df1c (127.6.96.1:42203)
I20260926 11:41:59.660902 18213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:59.657464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:58429 (local address 127.6.96.3:44065)
0926 11:41:59.657586 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:59.657592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:59.657815 (+ 223us) server_negotiation.cc:407] Connection header received
0926 11:41:59.658030 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:59.658033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:59.658082 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:59.658154 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:41:59.658421 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.658971 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.660015 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.660404 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.660493 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:59.660555 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:59.660710 (+ 155us) server_negotiation.cc:299] Negotiation successful
0926 11:41:59.660776 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 11:41:59.660921 18236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:59.657083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:44065 (local address 127.6.96.2:58429)
0926 11:41:59.657710 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:59.657725 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:59.657895 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:59.658176 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:59.658179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:59.658195 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:41:59.658315 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:59.658321 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.659084 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.659088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:59.659892 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:59.659900 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.660510 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.660514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:59.660603 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:59.660626 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 11:41:59.660690 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 11:41:59.660773 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"mutex_wait_us":2,"thread_start_us":79,"threads_started":1}
I20260926 11:41:59.661039 18217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:59.657522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:57833 (local address 127.6.96.1:42203)
0926 11:41:59.657637 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:59.657642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:59.657948 (+ 306us) server_negotiation.cc:407] Connection header received
0926 11:41:59.658070 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:59.658073 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:59.658114 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:59.658173 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:41:59.658490 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.659139 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.660151 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.660532 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.660619 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:59.660677 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:59.660842 (+ 165us) server_negotiation.cc:299] Negotiation successful
0926 11:41:59.660912 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 11:41:59.661056 18237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:59.657156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:42203 (local address 127.6.96.2:57833)
0926 11:41:59.657844 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:59.657856 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:41:59.657978 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:41:59.658263 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:41:59.658266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:41:59.658281 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:41:59.658384 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:59.658390 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.659248 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.659251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:59.660036 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:41:59.660044 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.660641 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.660644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:41:59.660738 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:59.660756 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 11:41:59.660816 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 11:41:59.660879 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"mutex_wait_us":18,"thread_start_us":101,"threads_started":1}
I20260926 11:41:59.661639 17882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36f637b3381145f88b83395c5561a4e0" candidate_uuid: "0f6a5181973f4384ab5fb77475523792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88bd63e4177244249ea720769108df1c" is_pre_election: true
I20260926 11:41:59.661664 18137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36f637b3381145f88b83395c5561a4e0" candidate_uuid: "0f6a5181973f4384ab5fb77475523792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ffbb7291796495b9b2a888aa73eec9f" is_pre_election: true
I20260926 11:41:59.661927 18137 raft_consensus.cc:2454] T 36f637b3381145f88b83395c5561a4e0 P 4ffbb7291796495b9b2a888aa73eec9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f6a5181973f4384ab5fb77475523792 in term 0.
I20260926 11:41:59.661926 17882 raft_consensus.cc:2454] T 36f637b3381145f88b83395c5561a4e0 P 88bd63e4177244249ea720769108df1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f6a5181973f4384ab5fb77475523792 in term 0.
I20260926 11:41:59.662343 17952 leader_election.cc:302] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f6a5181973f4384ab5fb77475523792, 4ffbb7291796495b9b2a888aa73eec9f; no voters:
I20260926 11:41:59.662621 18227 raft_consensus.cc:2781] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:41:59.662700 18227 raft_consensus.cc:482] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:41:59.662730 18227 raft_consensus.cc:3037] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:41:59.663717 18227 raft_consensus.cc:504] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffbb7291796495b9b2a888aa73eec9f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 44065 } } peers { permanent_uuid: "0f6a5181973f4384ab5fb77475523792" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 45757 } } peers { permanent_uuid: "88bd63e4177244249ea720769108df1c" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42203 } }
I20260926 11:41:59.664039 18227 leader_election.cc:288] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792 [CANDIDATE]: Term 1 election: Requested vote from peers 4ffbb7291796495b9b2a888aa73eec9f (127.6.96.3:44065), 88bd63e4177244249ea720769108df1c (127.6.96.1:42203)
I20260926 11:41:59.664384 17882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36f637b3381145f88b83395c5561a4e0" candidate_uuid: "0f6a5181973f4384ab5fb77475523792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88bd63e4177244249ea720769108df1c"
I20260926 11:41:59.664374 18137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36f637b3381145f88b83395c5561a4e0" candidate_uuid: "0f6a5181973f4384ab5fb77475523792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ffbb7291796495b9b2a888aa73eec9f"
I20260926 11:41:59.664496 18137 raft_consensus.cc:3037] T 36f637b3381145f88b83395c5561a4e0 P 4ffbb7291796495b9b2a888aa73eec9f [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:41:59.664495 17882 raft_consensus.cc:3037] T 36f637b3381145f88b83395c5561a4e0 P 88bd63e4177244249ea720769108df1c [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:41:59.665395 17882 raft_consensus.cc:2454] T 36f637b3381145f88b83395c5561a4e0 P 88bd63e4177244249ea720769108df1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f6a5181973f4384ab5fb77475523792 in term 1.
I20260926 11:41:59.665496 18137 raft_consensus.cc:2454] T 36f637b3381145f88b83395c5561a4e0 P 4ffbb7291796495b9b2a888aa73eec9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f6a5181973f4384ab5fb77475523792 in term 1.
I20260926 11:41:59.665657 17953 leader_election.cc:302] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f6a5181973f4384ab5fb77475523792, 88bd63e4177244249ea720769108df1c; no voters:
I20260926 11:41:59.665802 18227 raft_consensus.cc:2781] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:41:59.666000 18227 raft_consensus.cc:686] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792 [term 1 LEADER]: Becoming Leader. State: Replica: 0f6a5181973f4384ab5fb77475523792, State: Running, Role: LEADER
I20260926 11:41:59.666293 18227 consensus_queue.cc:237] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffbb7291796495b9b2a888aa73eec9f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 44065 } } peers { permanent_uuid: "0f6a5181973f4384ab5fb77475523792" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 45757 } } peers { permanent_uuid: "88bd63e4177244249ea720769108df1c" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42203 } }
I20260926 11:41:59.669185 17708 catalog_manager.cc:5495] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f6a5181973f4384ab5fb77475523792 (127.6.96.2). New cstate: current_term: 1 leader_uuid: "0f6a5181973f4384ab5fb77475523792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ffbb7291796495b9b2a888aa73eec9f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 44065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f6a5181973f4384ab5fb77475523792" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 45757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88bd63e4177244249ea720769108df1c" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42203 } health_report { overall_health: UNKNOWN } } }
I20260926 11:41:59.682111 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:41:59.679241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39286 (local address 127.6.96.2:45757)
0926 11:41:59.679348 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0926 11:41:59.679354 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 11:41:59.679370 (+ 16us) server_negotiation.cc:407] Connection header received
0926 11:41:59.679502 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:41:59.679504 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:41:59.679553 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:41:59.679635 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:41:59.679791 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.680263 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.680994 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:41:59.681185 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:41:59.681248 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:41:59.681455 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:41:59.681836 (+ 381us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:41:59.681922 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 11:41:59.681966 (+ 44us) server_negotiation.cc:299] Negotiation successful
0926 11:41:59.682012 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 11:41:59.687690 18137 raft_consensus.cc:1261] T 36f637b3381145f88b83395c5561a4e0 P 4ffbb7291796495b9b2a888aa73eec9f [term 1 FOLLOWER]: Refusing update from remote peer 0f6a5181973f4384ab5fb77475523792: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:41:59.687690 17882 raft_consensus.cc:1261] T 36f637b3381145f88b83395c5561a4e0 P 88bd63e4177244249ea720769108df1c [term 1 FOLLOWER]: Refusing update from remote peer 0f6a5181973f4384ab5fb77475523792: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:41:59.688385 18227 consensus_queue.cc:1035] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88bd63e4177244249ea720769108df1c" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:41:59.688575 18238 consensus_queue.cc:1035] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ffbb7291796495b9b2a888aa73eec9f" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 44065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:41:59.696167 18243 mvcc.cc:205] Tried to move back new op lower bound from 7333572279033446400 to 7333572278955692032. Current Snapshot: MvccSnapshot[applied={T|T < 7333572279033446400}]
I20260926 11:41:59.698119 18242 mvcc.cc:205] Tried to move back new op lower bound from 7333572279033446400 to 7333572278955692032. Current Snapshot: MvccSnapshot[applied={T|T < 7333572279033446400}]
I20260926 11:41:59.703220 17708 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:41:59.703404 17708 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:41:59.705672 17708 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0ca3bfd4f1c4a549c666d5a7c68eaea: Sending DeleteTablet for 3 replicas of tablet 36f637b3381145f88b83395c5561a4e0
I20260926 11:41:59.706362 17987 tablet_service.cc:1514] Processing DeleteTablet for tablet 36f637b3381145f88b83395c5561a4e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:41:59 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:39274
I20260926 11:41:59.706383 18132 tablet_service.cc:1514] Processing DeleteTablet for tablet 36f637b3381145f88b83395c5561a4e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:41:59 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:48394
I20260926 11:41:59.706404 17859 tablet_service.cc:1514] Processing DeleteTablet for tablet 36f637b3381145f88b83395c5561a4e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:41:59 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:41362
I20260926 11:41:59.706812 18251 tablet_replica.cc:330] T 36f637b3381145f88b83395c5561a4e0 P 88bd63e4177244249ea720769108df1c: stopping tablet replica
I20260926 11:41:59.706983 18253 tablet_replica.cc:330] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792: stopping tablet replica
I20260926 11:41:59.707208 18252 tablet_replica.cc:330] T 36f637b3381145f88b83395c5561a4e0 P 4ffbb7291796495b9b2a888aa73eec9f: stopping tablet replica
I20260926 11:41:59.707261 18251 raft_consensus.cc:2229] T 36f637b3381145f88b83395c5561a4e0 P 88bd63e4177244249ea720769108df1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:41:59.707376 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 17796
I20260926 11:41:59.708105 18253 raft_consensus.cc:2229] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792 [term 1 LEADER]: Raft consensus shutting down.
I20260926 11:41:59.708437 18253 raft_consensus.cc:2258] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:41:59.708722 18252 raft_consensus.cc:2229] T 36f637b3381145f88b83395c5561a4e0 P 4ffbb7291796495b9b2a888aa73eec9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:41:59.709050 18252 raft_consensus.cc:2258] T 36f637b3381145f88b83395c5561a4e0 P 4ffbb7291796495b9b2a888aa73eec9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:41:59.709863 18253 ts_tablet_manager.cc:1905] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:41:59.710558 18252 ts_tablet_manager.cc:1905] T 36f637b3381145f88b83395c5561a4e0 P 4ffbb7291796495b9b2a888aa73eec9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:41:59.712519 18253 ts_tablet_manager.cc:1918] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:41:59.712608 18253 log.cc:1196] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-1/wal/wals/36f637b3381145f88b83395c5561a4e0
I20260926 11:41:59.712899 18253 ts_tablet_manager.cc:1939] T 36f637b3381145f88b83395c5561a4e0 P 0f6a5181973f4384ab5fb77475523792: Deleting consensus metadata
I20260926 11:41:59.713358 18252 ts_tablet_manager.cc:1918] T 36f637b3381145f88b83395c5561a4e0 P 4ffbb7291796495b9b2a888aa73eec9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:41:59.713516 18252 log.cc:1196] T 36f637b3381145f88b83395c5561a4e0 P 4ffbb7291796495b9b2a888aa73eec9f: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/36f637b3381145f88b83395c5561a4e0
I20260926 11:41:59.713948 17701 catalog_manager.cc:4841] TS 0f6a5181973f4384ab5fb77475523792 (127.6.96.2:45757): tablet 36f637b3381145f88b83395c5561a4e0 (table test-table [id=c0fae4c5ad024619a10fc34847bdc6e4]) successfully deleted
I20260926 11:41:59.713857 18252 ts_tablet_manager.cc:1939] T 36f637b3381145f88b83395c5561a4e0 P 4ffbb7291796495b9b2a888aa73eec9f: Deleting consensus metadata
W20260926 11:41:59.714409 17700 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.96.1:42203 (error 108)
I20260926 11:41:59.715217 17699 catalog_manager.cc:4841] TS 4ffbb7291796495b9b2a888aa73eec9f (127.6.96.3:44065): tablet 36f637b3381145f88b83395c5561a4e0 (table test-table [id=c0fae4c5ad024619a10fc34847bdc6e4]) successfully deleted
W20260926 11:41:59.715334 18212 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:41:59.715048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:42203 (local address 127.0.0.1:41370)
0926 11:41:59.715165 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0926 11:41:59.715223 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.1:42203: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":38}
I20260926 11:41:59.715477 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 17935
W20260926 11:41:59.715528 17700 catalog_manager.cc:4584] TS 88bd63e4177244249ea720769108df1c (127.6.96.1:42203): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 36f637b3381145f88b83395c5561a4e0: Network error: Client connection negotiation failed: client connection to 127.6.96.1:42203: connect: Connection reset by peer (error 104)
I20260926 11:41:59.719982 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 18070
I20260926 11:41:59.724661 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 17683
I20260926 11:41:59.731343 6528 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 11:42:00.065465 6528 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 11:42:00.065591 6528 mini_ranger.cc:66] Stopping Ranger...
I20260926 11:42:00.399699 6528 mini_ranger.cc:68] Stopped Ranger
2026-09-26 11:42:00.399 UTC [17269] LOG: received smart shutdown request
2026-09-26 11:42:00.403 UTC [17269] LOG: background worker "logical replication launcher" (PID 17278) exited with exit code 1
2026-09-26 11:42:00.403 UTC [17273] LOG: shutting down
2026-09-26 11:42:00.550 UTC [17269] LOG: database system is shut down
2026-09-26T11:42:00Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (44678 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:42:00 dist-test-slave-xxxx krb5kdc[18263](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:42:00 dist-test-slave-xxxx krb5kdc[18263](info): set up 2 sockets
Sep 26 11:42:00 dist-test-slave-xxxx krb5kdc[18263](info): commencing operation
krb5kdc: starting...
W20260926 11:42:01.590684 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:42:01 dist-test-slave-xxxx krb5kdc[18263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:42:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:42:01Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.96.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.96.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.96.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/rangeradmin_127.6.96.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.96.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/rangeradmin_127.6.96.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.96.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.96.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.96.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/rangerlookup_127.6.96.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.96.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/rangerlookup_127.6.96.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.96.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.96.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.96.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/HTTP_127.6.96.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.96.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/HTTP_127.6.96.20@KRBTEST.COM.keytab.
I20260926 11:42:01.744728 6528 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskBunNuW/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/postgres -l logfile start
2026-09-26 11:42:04.103 UTC [18293] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 11:42:04.103 UTC [18293] LOG: listening on IPv4 address "127.6.96.20", port 56871
2026-09-26 11:42:04.106 UTC [18293] LOG: listening on Unix socket "/tmp/.s.PGSQL.56871"
2026-09-26 11:42:04.133 UTC [18296] LOG: database system was shut down at 2026-09-26 11:42:02 UTC
2026-09-26 11:42:04.137 UTC [18293] LOG: database system is ready to accept connections
I20260926 11:42:05.002424 6528 mini_postgres.cc:96] Postgres bound to 56871
2026-09-26 11:42:05.008 UTC [18304] FATAL: database "slave" does not exist
127.6.96.20:56871 - accepting connections
I20260926 11:42:05.008651 6528 mini_ranger.cc:161] Starting Ranger...
I20260926 11:42:05.022935 6528 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 11:42:05.484783 6528 mini_ranger.cc:87] Created ranger Postgres database
I20260926 11:42:05.484850 6528 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ranger-admin
2026-09-26 11:42:05,757 [I] DB FLAVOR :POSTGRES
2026-09-26 11:42:05,757 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 11:42:05,757 [I] Checking connection..
2026-09-26 11:42:05,757 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 11:42:06,014 [I] Checking connection passed.
2026-09-26 11:42:06,015 [I] --------- Verifying version history table ---------
2026-09-26 11:42:06,015 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 11:42:06,257 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 11:42:06,257 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 11:42:06,496 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 11:42:06,496 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 11:42:06,496 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 11:42:06.744 UTC [18368] WARNING: there is no transaction in progress
2026-09-26 11:42:06,749 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 11:42:06,750 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 11:42:07,005 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 11:42:07,006 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 11:42:07,006 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 11:42:07,252 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 11:42:07,503 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-xxxx.c.gcp-upstream.internal','N') ;"
2026-09-26 11:42:07,747 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 11:42:07,994 [I] Table x_portal_user does not exist in database ranger
2026-09-26 11:42:07,994 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 11:42:08,230 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 11:42:08,230 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 11:42:08,456 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 11:42:08,689 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 11:42:08,689 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 11:42:09.657 UTC [18476] WARNING: there is no transaction in progress
2026-09-26 11:42:09.674 UTC [18476] WARNING: there is no transaction in progress
2026-09-26 11:42:09.689 UTC [18476] WARNING: there is no transaction in progress
2026-09-26 11:42:09.704 UTC [18476] WARNING: there is no transaction in progress
2026-09-26 11:42:09.723 UTC [18476] WARNING: there is no transaction in progress
2026-09-26 11:42:09.740 UTC [18476] WARNING: there is no transaction in progress
2026-09-26 11:42:09.796 UTC [18476] WARNING: there is no transaction in progress
2026-09-26 11:42:09.803 UTC [18476] WARNING: there is no transaction in progress
2026-09-26 11:42:09.817 UTC [18476] WARNING: there is no transaction in progress
2026-09-26 11:42:09.827 UTC [18476] WARNING: there is no transaction in progress
2026-09-26 11:42:09.837 UTC [18476] WARNING: there is no transaction in progress
2026-09-26 11:42:09.851 UTC [18476] WARNING: there is no transaction in progress
2026-09-26 11:42:09.861 UTC [18476] WARNING: there is no transaction in progress
2026-09-26 11:42:09.867 UTC [18476] WARNING: there is no transaction in progress
2026-09-26 11:42:09.874 UTC [18476] WARNING: there is no transaction in progress
2026-09-26 11:42:09.879 UTC [18476] WARNING: there is no transaction in progress
2026-09-26 11:42:10,498 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 11:42:10,499 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 11:42:10,731 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 11:42:10,731 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 11:42:10,961 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 11:42:10,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 11:42:11,189 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 11:42:11,189 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 11:42:11,432 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 11:42:11,665 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-xxxx.c.gcp-upstream.internal';"
2026-09-26 11:42:11,906 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 11:42:11,906 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 11:42:12,139 [I] DB_PATCHES have already been applied
I20260926 11:42:12.144958 6528 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ranger-admin:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskBunNuW/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 11:42:12.145021 6528 mini_ranger.cc:193] Using host: 127.6.96.20
I20260926 11:42:12.148093 6528 mini_ranger.cc:227] Ranger admin URL: http://127.6.96.20:45195
Sep 26, 2026 11:42:12 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 11:42:12 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 11:42:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 11:42:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 11:42:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 11:42:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 11:42:12 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 11:42:13.152851 6528 mini_ranger.cc:160] Time spent starting Ranger: real 8.144s user 0.005s sys 0.003s
Sep 26, 2026 11:42:32 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 11:42:32 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 26, 2026 11:42:32 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 11:42:32 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 11:42:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 11:42:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 11:42:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 11:42:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 11:42:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 11:42:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 11:42:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 11:42:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 11:42:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 11:42:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 11:42:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 11:42:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 11:42:32 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 11:42:33 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 11:42:34.371825 6528 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.96.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.96.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.96.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/rangerkms_127.6.96.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.96.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/rangerkms_127.6.96.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.96.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.96.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.96.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/HTTP_127.6.96.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.96.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/HTTP_127.6.96.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 11:42:34 dist-test-slave-xxxx krb5kdc[18263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422954, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 11:42:34.472520 6528 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 11:42:34.481563 6528 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 11:42:35.308348 6528 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 11:42:35.308430 6528 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ranger-kms
I20260926 11:42:35.308446 6528 mini_ranger_kms.cc:227] Using postgres at 127.6.96.20:56871
2026-09-26 11:42:35,465 [I] DB FLAVOR :POSTGRES
2026-09-26 11:42:35,465 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 11:42:35,465 [I] Checking connection
2026-09-26 11:42:35,465 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:56871/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 11:42:35,694 [I] connection success
2026-09-26 11:42:35,694 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 11:42:35,694 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 11:42:35,694 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:56871/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 11:42:35,958 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 11:42:35,959 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 11:42:35,959 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 11:42:35,959 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.96.20:56871/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 11:42:36,237 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 11:42:36.704469 6528 mini_ranger_kms.cc:310] Created kms service
I20260926 11:42:36.928439 6528 mini_ranger_kms.cc:326] Created kudu user
I20260926 11:42:36.970202 6528 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 11:42:36 dist-test-slave-xxxx krb5kdc[18263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422954, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.96.20@KRBTEST.COM
I20260926 11:42:37.380509 6528 mini_ranger_kms.cc:384] Added ranger policy
I20260926 11:42:37.380777 6528 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ranger-kms:/tmp/dist-test-taskBunNuW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskBunNuW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 11:42:37.380807 6528 mini_ranger_kms.cc:240] Using host: 127.6.96.20
I20260926 11:42:37.384191 6528 mini_ranger_kms.cc:276] Ranger KMS PID: 18656
I20260926 11:42:37.384271 6528 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.96.20:34403
Sep 26, 2026 11:42:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 11:42:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 11:42:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 11:42:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 11:42:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 11:42:38 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 11:42:39.442533 6528 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.970s user 0.020s sys 0.120s
I20260926 11:42:39.442656 6528 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 11:42:39.442731 6528 mini_ranger_kms.cc:401] 127.6.96.20:34403/kms/v1/keys
Sep 26 11:42:39 dist-test-slave-xxxx krb5kdc[18263](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790422959, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.96.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 11:42:40 dist-test-slave-xxxx krb5kdc[18263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422959, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.96.20@KRBTEST.COM for HTTP/127.6.96.20@KRBTEST.COM
Sep 26 11:42:40 dist-test-slave-xxxx krb5kdc[18263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422959, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.96.20@KRBTEST.COM for HTTP/127.6.96.20@KRBTEST.COM
Sep 26 11:42:42 dist-test-slave-xxxx krb5kdc[18263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:42:42.536203 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:36855
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:36855
--ranger_config_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.96.20:34403/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:42:42.644186 18707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:42:42.644450 18707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:42:42.644488 18707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:42:42.649561 18707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:42:42.649631 18707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:42:42.649649 18707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:42:42.649664 18707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:42:42.649677 18707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:42:42.655637 18707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.96.20:34403/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:36855
--ranger_config_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:36855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:42:42.656715 18707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:42:42.657724 18707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:42:42.666165 18713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:42.666307 18707 server_base.cc:1034] running on GCE node
W20260926 11:42:42.666201 18712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:42.666221 18715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:42.667145 18707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:42:42.668239 18707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:42:42.669401 18707 hybrid_clock.cc:648] HybridClock initialized: now 1790422962669375 us; error 38 us; skew 500 ppm
Sep 26 11:42:42 dist-test-slave-xxxx krb5kdc[18263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422962, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:42.681156 18707 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:42:42.682169 18707 webserver.cc:460] Webserver started at http://127.6.96.62:39697/ using document root <none> and password file <none>
I20260926 11:42:42.682744 18707 fs_manager.cc:359] Metadata directory not provided
I20260926 11:42:42.682804 18707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:42:42.683029 18707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 11:42:42 dist-test-slave-xxxx krb5kdc[18263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422962, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for HTTP/127.6.96.20@KRBTEST.COM
I20260926 11:42:42.734463 18707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "bf07eb7b624846cb9d9a677c82057a12"
format_stamp: "Formatted at 2026-09-26 11:42:42 on dist-test-slave-xxxx"
server_key: "ed93d9e71af900b6461a856ec57ffd80"
server_key_iv: "baf614437a866eb006d8e04756b94bfb"
server_key_version: "kuduclusterkey@0"
I20260926 11:42:42.735222 18707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "bf07eb7b624846cb9d9a677c82057a12"
format_stamp: "Formatted at 2026-09-26 11:42:42 on dist-test-slave-xxxx"
server_key: "ed93d9e71af900b6461a856ec57ffd80"
server_key_iv: "baf614437a866eb006d8e04756b94bfb"
server_key_version: "kuduclusterkey@0"
I20260926 11:42:42.739318 18707 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 11:42:42.755203 18722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:42.756554 18707 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 11:42:42.756686 18707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "bf07eb7b624846cb9d9a677c82057a12"
format_stamp: "Formatted at 2026-09-26 11:42:42 on dist-test-slave-xxxx"
server_key: "ed93d9e71af900b6461a856ec57ffd80"
server_key_iv: "baf614437a866eb006d8e04756b94bfb"
server_key_version: "kuduclusterkey@0"
I20260926 11:42:42.756821 18707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:42:42.771224 18707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:42:42.774552 18707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:42:42.774731 18707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:42:42.781623 18774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:36855 every 8 connection(s)
I20260926 11:42:42.781623 18707 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:36855
I20260926 11:42:42.782637 18707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:42:42.783092 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 18707
I20260926 11:42:42.783207 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:42:42.783504 6528 external_mini_cluster.cc:1428] Setting key c7b9f3cd30d32a9c6c30af44ef55d7aa
I20260926 11:42:42.785740 18775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:42:42.792188 18775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12: Bootstrap starting.
Sep 26 11:42:42 dist-test-slave-xxxx krb5kdc[18263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:42:42.794646 18775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12: Neither blocks nor log segments found. Creating new log.
I20260926 11:42:42.795907 18775 log.cc:824] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12: Log is configured to *not* fsync() on all Append() calls
I20260926 11:42:42.798028 18778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:42.785254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46616 (local address 127.6.96.62:36855)
0926 11:42:42.785672 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:42.785686 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:42.785723 (+ 37us) server_negotiation.cc:407] Connection header received
0926 11:42:42.786376 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:42.786393 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:42.786724 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:42.787051 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:42.787863 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:42.788838 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:42.789957 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:42.790503 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:42.790643 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:42.793310 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:42.793317 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:42.793333 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:42.793363 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:42.795110 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:42.795509 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:42.795514 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:42.795519 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:42.795650 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:42.795951 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:42.795954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:42.795956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:42.796427 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:42.796575 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:42.796886 (+ 311us) server_negotiation.cc:299] Negotiation successful
0926 11:42:42.797143 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":134,"threads_started":1}
I20260926 11:42:42.798032 18775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12: No bootstrap required, opened a new log
I20260926 11:42:42.801201 18775 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf07eb7b624846cb9d9a677c82057a12" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36855 } }
I20260926 11:42:42.801409 18775 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:42:42.801463 18775 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf07eb7b624846cb9d9a677c82057a12, State: Initialized, Role: FOLLOWER
I20260926 11:42:42.801887 18775 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf07eb7b624846cb9d9a677c82057a12" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36855 } }
I20260926 11:42:42.802009 18775 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:42:42.802054 18775 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:42:42.802129 18775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:42:42.802932 18775 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf07eb7b624846cb9d9a677c82057a12" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36855 } }
I20260926 11:42:42.803215 18775 leader_election.cc:302] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf07eb7b624846cb9d9a677c82057a12; no voters:
I20260926 11:42:42.803444 18775 leader_election.cc:288] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:42:42.803546 18780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:42:42.803747 18780 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [term 1 LEADER]: Becoming Leader. State: Replica: bf07eb7b624846cb9d9a677c82057a12, State: Running, Role: LEADER
I20260926 11:42:42.804076 18780 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf07eb7b624846cb9d9a677c82057a12" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36855 } }
I20260926 11:42:42.804493 18775 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:42:42.805485 18781 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf07eb7b624846cb9d9a677c82057a12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf07eb7b624846cb9d9a677c82057a12" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36855 } } }
I20260926 11:42:42.805612 18781 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [sys.catalog]: This master's current role is: LEADER
I20260926 11:42:42.805881 18782 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf07eb7b624846cb9d9a677c82057a12. Latest consensus state: current_term: 1 leader_uuid: "bf07eb7b624846cb9d9a677c82057a12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf07eb7b624846cb9d9a677c82057a12" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36855 } } }
I20260926 11:42:42.805959 18789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:42:42.805963 18782 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [sys.catalog]: This master's current role is: LEADER
I20260926 11:42:42.808648 18789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:42:42.809032 18775 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 11:42:42.816792 18789 catalog_manager.cc:1347] Generated new cluster ID: 488e75bdc61448168f264bbe0e9c295c
I20260926 11:42:42.816931 18789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:42:42.836431 18789 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:42:42.838088 18789 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:42:42.843281 18789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12: Generated new TSK 0
I20260926 11:42:42.844442 18789 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.96.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
I20260926 11:42:43.528352 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.96.20:34403/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:36855
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:42:43.665917 18820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:42:43.666217 18820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:42:43.666276 18820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:42:43.670971 18820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:42:43.671061 18820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:42:43.671149 18820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:42:43.679159 18820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.96.20:34403/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:36855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:42:43.680553 18820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:42:43.681845 18820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:42:43.692564 18827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:43.692564 18830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:43.693368 18828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:43.693578 18820 server_base.cc:1034] running on GCE node
I20260926 11:42:43.693985 18820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:42:43.694658 18820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:42:43.695828 18820 hybrid_clock.cc:648] HybridClock initialized: now 1790422963695794 us; error 55 us; skew 500 ppm
Sep 26 11:42:43 dist-test-slave-xxxx krb5kdc[18263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422963, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:43.699296 18820 init.cc:377] Logged in from keytab as kudu/127.6.96.1@KRBTEST.COM (short username kudu)
I20260926 11:42:43.700459 18820 webserver.cc:460] Webserver started at http://127.6.96.1:34809/ using document root <none> and password file <none>
I20260926 11:42:43.701172 18820 fs_manager.cc:359] Metadata directory not provided
I20260926 11:42:43.701264 18820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:42:43.701524 18820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 11:42:43 dist-test-slave-xxxx krb5kdc[18263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422963, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for HTTP/127.6.96.20@KRBTEST.COM
I20260926 11:42:43.725769 18820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "7a202cf1ada7405a9775e9265acc2c08"
format_stamp: "Formatted at 2026-09-26 11:42:43 on dist-test-slave-xxxx"
server_key: "227e6997360ed5fb9ef6cd164d4d0239"
server_key_iv: "8ea51b214477598da3c7de152a479a01"
server_key_version: "kuduclusterkey@0"
I20260926 11:42:43.727706 18820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "7a202cf1ada7405a9775e9265acc2c08"
format_stamp: "Formatted at 2026-09-26 11:42:43 on dist-test-slave-xxxx"
server_key: "227e6997360ed5fb9ef6cd164d4d0239"
server_key_iv: "8ea51b214477598da3c7de152a479a01"
server_key_version: "kuduclusterkey@0"
I20260926 11:42:43.732919 18820 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 11:42:43.763646 18839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:43.765324 18820 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:42:43.765455 18820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "7a202cf1ada7405a9775e9265acc2c08"
format_stamp: "Formatted at 2026-09-26 11:42:43 on dist-test-slave-xxxx"
server_key: "227e6997360ed5fb9ef6cd164d4d0239"
server_key_iv: "8ea51b214477598da3c7de152a479a01"
server_key_version: "kuduclusterkey@0"
I20260926 11:42:43.765594 18820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:42:43.804745 18820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:42:43.808635 18820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:42:43.808827 18820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:42:43.809583 18820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:42:43.810608 18820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:42:43.810670 18820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:43.810719 18820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:42:43.810746 18820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 11:42:43 dist-test-slave-xxxx krb5kdc[18263](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790422963, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:43.821008 18820 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:35883
I20260926 11:42:43.821048 18952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:35883 every 8 connection(s)
I20260926 11:42:43.822232 18820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:42:43.829432 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 18820
I20260926 11:42:43.829573 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:42:43.829856 6528 external_mini_cluster.cc:1428] Setting key 085443bd1c24ffd1b4dce73c67672813
WARNING: no policy specified for kudu/127.6.96.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.2@KRBTEST.COM" created.
Sep 26 11:42:43 dist-test-slave-xxxx krb5kdc[18263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422963, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:42:43.853967 18956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:43.826858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:48017 (local address 127.6.96.62:36855)
0926 11:42:43.839059 (+ 12201us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:43.839070 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:43.839093 (+ 23us) server_negotiation.cc:407] Connection header received
0926 11:42:43.839158 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:43.839162 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:43.839224 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:43.839355 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:43.841084 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:43.841880 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:43.843660 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:43.843972 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:43.844047 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:43.849166 (+ 5119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:43.849170 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:43.849173 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:43.849203 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:43.851141 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:43.851853 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:43.851857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:43.851860 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:43.851985 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:43.852458 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:43.852462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:43.852464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:43.852726 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:43.852821 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:43.853607 (+ 786us) server_negotiation.cc:299] Negotiation successful
0926 11:42:43.853757 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12090,"thread_start_us":86,"threads_started":1}
I20260926 11:42:43.855722 18955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:43.825704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:36855 (local address 127.6.96.1:48017)
0926 11:42:43.827420 (+ 1716us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:43.827481 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:43.828564 (+ 1083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:43.839653 (+ 11089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:43.839668 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:43.840087 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:42:43.840866 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:43.840890 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:43.842030 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:43.842034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:43.843494 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:43.843507 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:43.844258 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:43.844262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:43.844472 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:43.847027 (+ 2555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:42:43.847063 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:42:43.848914 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:42:43.851307 (+ 2393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:43.851315 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:43.851335 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:43.851714 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:43.852111 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:43.852114 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:43.852117 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:43.852333 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:43.852896 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:42:43.852904 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:42:43.853290 (+ 386us) client_negotiation.cc:769] Sending connection context
0926 11:42:43.854231 (+ 941us) client_negotiation.cc:240] Negotiation successful
0926 11:42:43.854616 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1458,"thread_start_us":124,"threads_started":1}
I20260926 11:42:43.857081 18953 heartbeater.cc:347] Connected to a master server at 127.6.96.62:36855
I20260926 11:42:43.857318 18953 heartbeater.cc:464] Registering TS with master...
I20260926 11:42:43.857852 18953 heartbeater.cc:511] Master 127.6.96.62:36855 requested a full tablet report, sending...
I20260926 11:42:43.859475 18732 ts_manager.cc:194] Registered new tserver with Master: 7a202cf1ada7405a9775e9265acc2c08 (127.6.96.1:35883)
I20260926 11:42:43.861003 18732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.1@KRBTEST.COM'} at 127.6.96.1:48017
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
I20260926 11:42:43.869834 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.96.20:34403/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:36855
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:42:44.030273 18959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:42:44.030695 18959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:42:44.030776 18959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:42:44.036964 18959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:42:44.037108 18959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:42:44.037216 18959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:42:44.045295 18959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.96.20:34403/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:36855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:42:44.046979 18959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:42:44.048448 18959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:42:44.061365 18965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:44.061524 18968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:44.061959 18966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:44.063119 18959 server_base.cc:1034] running on GCE node
I20260926 11:42:44.063618 18959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:42:44.064440 18959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:42:44.067745 18959 hybrid_clock.cc:648] HybridClock initialized: now 1790422964067667 us; error 90 us; skew 500 ppm
Sep 26 11:42:44 dist-test-slave-xxxx krb5kdc[18263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422964, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:44.071514 18959 init.cc:377] Logged in from keytab as kudu/127.6.96.2@KRBTEST.COM (short username kudu)
I20260926 11:42:44.072600 18959 webserver.cc:460] Webserver started at http://127.6.96.2:38277/ using document root <none> and password file <none>
I20260926 11:42:44.073149 18959 fs_manager.cc:359] Metadata directory not provided
I20260926 11:42:44.073212 18959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:42:44.073411 18959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 11:42:44 dist-test-slave-xxxx krb5kdc[18263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422964, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for HTTP/127.6.96.20@KRBTEST.COM
I20260926 11:42:44.089032 18959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "08855d2e07084434b67250db07813e81"
format_stamp: "Formatted at 2026-09-26 11:42:44 on dist-test-slave-xxxx"
server_key: "70815587b211b291396dd756249542d5"
server_key_iv: "1611703744f8434aca482fcc32cfaf6f"
server_key_version: "kuduclusterkey@0"
I20260926 11:42:44.089820 18959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "08855d2e07084434b67250db07813e81"
format_stamp: "Formatted at 2026-09-26 11:42:44 on dist-test-slave-xxxx"
server_key: "70815587b211b291396dd756249542d5"
server_key_iv: "1611703744f8434aca482fcc32cfaf6f"
server_key_version: "kuduclusterkey@0"
I20260926 11:42:44.094803 18959 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260926 11:42:44.124251 18975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:44.128191 18959 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.002s sys 0.002s
I20260926 11:42:44.128364 18959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "08855d2e07084434b67250db07813e81"
format_stamp: "Formatted at 2026-09-26 11:42:44 on dist-test-slave-xxxx"
server_key: "70815587b211b291396dd756249542d5"
server_key_iv: "1611703744f8434aca482fcc32cfaf6f"
server_key_version: "kuduclusterkey@0"
I20260926 11:42:44.128523 18959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:42:44.150779 18959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:42:44.153846 18959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:42:44.154026 18959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:42:44.154670 18959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:42:44.155646 18959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:42:44.155694 18959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:44.155740 18959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:42:44.155769 18959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:44.165997 18959 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:35867
I20260926 11:42:44.167165 18959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:42:44.171249 19088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:35867 every 8 connection(s)
I20260926 11:42:44.175072 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 18959
I20260926 11:42:44.175199 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:42:44.175523 6528 external_mini_cluster.cc:1428] Setting key 5aab7fad983b98bb1347fd7c0ebf68ff
Sep 26 11:42:44 dist-test-slave-xxxx krb5kdc[18263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422964, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:42:44.186697 18956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:44.173242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:47471 (local address 127.6.96.62:36855)
0926 11:42:44.173372 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:44.173377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:44.174353 (+ 976us) server_negotiation.cc:407] Connection header received
0926 11:42:44.175606 (+ 1253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:44.175610 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:44.175668 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:44.175775 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:44.177629 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:44.178363 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:44.179665 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:44.180052 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:44.180134 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:44.182894 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:44.182898 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:44.182901 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:44.182929 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:44.184608 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:44.185111 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:44.185114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:44.185116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:44.185195 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:44.185529 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:44.185533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:44.185534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:44.185717 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:44.185797 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:44.186373 (+ 576us) server_negotiation.cc:299] Negotiation successful
0926 11:42:44.186513 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 11:42:44.187623 19091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:44.173704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:36855 (local address 127.6.96.2:47471)
0926 11:42:44.174195 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:44.174249 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:44.175360 (+ 1111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:44.175980 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:44.175989 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:44.176785 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:42:44.177468 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:44.177487 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:44.178521 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:44.178525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:44.179530 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:44.179542 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:44.180390 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:44.180395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:44.180601 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:44.181231 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:42:44.181252 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:42:44.182705 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:42:44.184726 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:44.184732 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:44.184752 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:44.185020 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:44.185285 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:44.185288 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:44.185290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:44.185437 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:44.185812 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:42:44.185819 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:42:44.186135 (+ 316us) client_negotiation.cc:769] Sending connection context
0926 11:42:44.186310 (+ 175us) client_negotiation.cc:240] Negotiation successful
0926 11:42:44.186633 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":120,"threads_started":1}
I20260926 11:42:44.188848 19089 heartbeater.cc:347] Connected to a master server at 127.6.96.62:36855
I20260926 11:42:44.189087 19089 heartbeater.cc:464] Registering TS with master...
I20260926 11:42:44.189601 19089 heartbeater.cc:511] Master 127.6.96.62:36855 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.96.3@KRBTEST.COM; defaulting to no policy
I20260926 11:42:44.190608 18732 ts_manager.cc:194] Registered new tserver with Master: 08855d2e07084434b67250db07813e81 (127.6.96.2:35867)
I20260926 11:42:44.191334 18732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.2@KRBTEST.COM'} at 127.6.96.2:47471
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
I20260926 11:42:44.216655 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.96.20:34403/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:36855
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 11:42:44 dist-test-slave-xxxx krb5kdc[18263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422963, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for HTTP/127.6.96.20@KRBTEST.COM
W20260926 11:42:44.381696 19094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:42:44.382041 19094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:42:44.382108 19094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:42:44.388240 19094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:42:44.388362 19094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:42:44.388461 19094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:42:44.396031 19094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.96.20:34403/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:36855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:42:44.397367 19094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:42:44.398749 19094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:42:44.410537 19102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:44.410657 19100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:44.411033 19099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:44.411305 19094 server_base.cc:1034] running on GCE node
I20260926 11:42:44.411779 19094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:42:44.412629 19094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:42:44.413829 19094 hybrid_clock.cc:648] HybridClock initialized: now 1790422964413795 us; error 45 us; skew 500 ppm
Sep 26 11:42:44 dist-test-slave-xxxx krb5kdc[18263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422964, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:44.417416 19094 init.cc:377] Logged in from keytab as kudu/127.6.96.3@KRBTEST.COM (short username kudu)
I20260926 11:42:44.418470 19094 webserver.cc:460] Webserver started at http://127.6.96.3:46103/ using document root <none> and password file <none>
I20260926 11:42:44.419199 19094 fs_manager.cc:359] Metadata directory not provided
I20260926 11:42:44.419279 19094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:42:44.419485 19094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 11:42:44 dist-test-slave-xxxx krb5kdc[18263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422964, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for HTTP/127.6.96.20@KRBTEST.COM
I20260926 11:42:44.436031 19094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "ec3eb9d834e5443a91c605891cb036b3"
format_stamp: "Formatted at 2026-09-26 11:42:44 on dist-test-slave-xxxx"
server_key: "388286b6776fbcbc12af94ee9bed2292"
server_key_iv: "0aa7df5dce08c2911257e50b37e044bf"
server_key_version: "kuduclusterkey@0"
I20260926 11:42:44.436764 19094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "ec3eb9d834e5443a91c605891cb036b3"
format_stamp: "Formatted at 2026-09-26 11:42:44 on dist-test-slave-xxxx"
server_key: "388286b6776fbcbc12af94ee9bed2292"
server_key_iv: "0aa7df5dce08c2911257e50b37e044bf"
server_key_version: "kuduclusterkey@0"
I20260926 11:42:44.447813 19094 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.005s sys 0.000s
I20260926 11:42:44.467208 19109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:44.468971 19094 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260926 11:42:44.469110 19094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "ec3eb9d834e5443a91c605891cb036b3"
format_stamp: "Formatted at 2026-09-26 11:42:44 on dist-test-slave-xxxx"
server_key: "388286b6776fbcbc12af94ee9bed2292"
server_key_iv: "0aa7df5dce08c2911257e50b37e044bf"
server_key_version: "kuduclusterkey@0"
I20260926 11:42:44.469262 19094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:42:44.500566 19094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:42:44.504118 19094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:42:44.504314 19094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:42:44.505098 19094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:42:44.506168 19094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:42:44.506228 19094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:44.506281 19094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:42:44.506311 19094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:44.516824 19094 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:44035
I20260926 11:42:44.517045 19222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:44035 every 8 connection(s)
I20260926 11:42:44.518005 19094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:42:44.522281 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 19094
I20260926 11:42:44.522409 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:42:44.522725 6528 external_mini_cluster.cc:1428] Setting key 12a8ac9c5d4596963885bec4b1c708b8
Sep 26 11:42:44 dist-test-slave-xxxx krb5kdc[18263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422964, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:42:44.535637 18956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:44.520196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:60283 (local address 127.6.96.62:36855)
0926 11:42:44.520324 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:44.520329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:44.521478 (+ 1149us) server_negotiation.cc:407] Connection header received
0926 11:42:44.522821 (+ 1343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:44.522848 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:44.522906 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:44.522954 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:44.525487 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:44.526317 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:44.527994 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:44.528349 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:44.528462 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:44.531340 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:44.531344 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:44.531347 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:44.531377 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:44.532976 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:44.533627 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:44.533631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:44.533633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:44.533733 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:44.534210 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:44.534214 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:44.534216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:44.534454 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:44.534552 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:44.535323 (+ 771us) server_negotiation.cc:299] Negotiation successful
0926 11:42:44.535459 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 11:42:44.538266 19225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:44.520746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:36855 (local address 127.6.96.3:60283)
0926 11:42:44.521306 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:44.521364 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:44.522525 (+ 1161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:44.523723 (+ 1198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:44.523733 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:44.524066 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:42:44.525326 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:44.525347 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:44.526443 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:44.526447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:44.527858 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:44.527868 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:44.528516 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:44.528520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:44.528709 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:44.529382 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:42:44.529404 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:42:44.531148 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:42:44.533136 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:44.533143 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:44.533162 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:44.533514 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:44.533826 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:44.533829 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:44.533831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:44.534091 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:44.534551 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:42:44.534558 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:42:44.534957 (+ 399us) client_negotiation.cc:769] Sending connection context
0926 11:42:44.536401 (+ 1444us) client_negotiation.cc:240] Negotiation successful
0926 11:42:44.537029 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":165,"threads_started":1}
I20260926 11:42:44.539907 19223 heartbeater.cc:347] Connected to a master server at 127.6.96.62:36855
I20260926 11:42:44.540148 19223 heartbeater.cc:464] Registering TS with master...
I20260926 11:42:44.540727 19223 heartbeater.cc:511] Master 127.6.96.62:36855 requested a full tablet report, sending...
I20260926 11:42:44.541865 18732 ts_manager.cc:194] Registered new tserver with Master: ec3eb9d834e5443a91c605891cb036b3 (127.6.96.3:44035)
I20260926 11:42:44.542582 18732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.3@KRBTEST.COM'} at 127.6.96.3:60283
I20260926 11:42:44.548858 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 11:42:44.549821 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 18820
I20260926 11:42:44.556686 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 18959
I20260926 11:42:44.563395 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 19094
I20260926 11:42:44.570154 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 18707
I20260926 11:42:44.575811 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:36855
--webserver_interface=127.6.96.62
--webserver_port=39697
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:36855
--ranger_config_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.96.20:34403/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.96.20:34403/kms with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:42:44.685828 19227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:42:44.686129 19227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:42:44.686169 19227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:42:44.690279 19227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:42:44.690346 19227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:42:44.690364 19227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:42:44.690378 19227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:42:44.690392 19227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:42:44.696053 19227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.96.20:34403/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:36855
--ranger_config_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:36855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=39697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:42:44.697135 19227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:42:44.698166 19227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:42:44.706634 19233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:44.706660 19232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:44.706634 19235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:44.706921 19227 server_base.cc:1034] running on GCE node
I20260926 11:42:44.707484 19227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:42:44.708596 19227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:42:44.709759 19227 hybrid_clock.cc:648] HybridClock initialized: now 1790422964709736 us; error 41 us; skew 500 ppm
Sep 26 11:42:44 dist-test-slave-xxxx krb5kdc[18263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422964, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:44.712546 19227 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:42:44.713420 19227 webserver.cc:460] Webserver started at http://127.6.96.62:39697/ using document root <none> and password file <none>
I20260926 11:42:44.713932 19227 fs_manager.cc:359] Metadata directory not provided
I20260926 11:42:44.714010 19227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 11:42:44 dist-test-slave-xxxx krb5kdc[18263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422964, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for HTTP/127.6.96.20@KRBTEST.COM
I20260926 11:42:44.784649 19227 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:42:44.787086 19243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:44.788483 19227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:42:44.788614 19227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "bf07eb7b624846cb9d9a677c82057a12"
format_stamp: "Formatted at 2026-09-26 11:42:42 on dist-test-slave-xxxx"
server_key: "ed93d9e71af900b6461a856ec57ffd80"
server_key_iv: "baf614437a866eb006d8e04756b94bfb"
server_key_version: "kuduclusterkey@0"
I20260926 11:42:44.789176 19227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:42:44.818297 19227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:42:44.821259 19227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:42:44.821507 19227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:42:44.828694 19227 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:36855
I20260926 11:42:44.828720 19295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:36855 every 8 connection(s)
I20260926 11:42:44.829759 19227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:42:44.833622 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 19227
I20260926 11:42:44.833979 19296 sys_catalog.cc:263] Verifying existing consensus state
I20260926 11:42:44.834113 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.96.20:34403/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:35883
--local_ip_for_outbound_sockets=127.6.96.1
--tserver_master_addrs=127.6.96.62:36855
--webserver_port=34809
--webserver_interface=127.6.96.1
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.96.20:34403/kms with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 11:42:44.835961 19296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12: Bootstrap starting.
I20260926 11:42:44.851749 19296 log.cc:824] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12: Log is configured to *not* fsync() on all Append() calls
I20260926 11:42:44.858098 19296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 11:42:44.858448 19296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12: Bootstrap complete.
I20260926 11:42:44.862310 19296 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf07eb7b624846cb9d9a677c82057a12" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36855 } }
I20260926 11:42:44.862707 19296 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf07eb7b624846cb9d9a677c82057a12, State: Initialized, Role: FOLLOWER
I20260926 11:42:44.863175 19296 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf07eb7b624846cb9d9a677c82057a12" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36855 } }
I20260926 11:42:44.863288 19296 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:42:44.863345 19296 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:42:44.863440 19296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [term 1 FOLLOWER]: Advancing to term 2
I20260926 11:42:44.865231 19296 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf07eb7b624846cb9d9a677c82057a12" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36855 } }
I20260926 11:42:44.865537 19296 leader_election.cc:302] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf07eb7b624846cb9d9a677c82057a12; no voters:
I20260926 11:42:44.865803 19296 leader_election.cc:288] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 11:42:44.865935 19300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 11:42:44.866189 19300 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [term 2 LEADER]: Becoming Leader. State: Replica: bf07eb7b624846cb9d9a677c82057a12, State: Running, Role: LEADER
I20260926 11:42:44.866508 19300 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf07eb7b624846cb9d9a677c82057a12" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36855 } }
I20260926 11:42:44.866952 19296 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:42:44.867859 19301 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bf07eb7b624846cb9d9a677c82057a12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf07eb7b624846cb9d9a677c82057a12" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36855 } } }
I20260926 11:42:44.867990 19301 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [sys.catalog]: This master's current role is: LEADER
I20260926 11:42:44.868283 19302 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf07eb7b624846cb9d9a677c82057a12. Latest consensus state: current_term: 2 leader_uuid: "bf07eb7b624846cb9d9a677c82057a12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf07eb7b624846cb9d9a677c82057a12" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36855 } } }
I20260926 11:42:44.868359 19302 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12 [sys.catalog]: This master's current role is: LEADER
I20260926 11:42:44.868814 19309 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:42:44.869099 19296 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 11:42:44.875119 19309 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:42:44.876183 19309 catalog_manager.cc:1259] Loaded cluster ID: 488e75bdc61448168f264bbe0e9c295c
I20260926 11:42:44.876289 19309 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:42:44.879413 19309 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:42:44.880158 19309 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12: Loaded TSK: 0
I20260926 11:42:44.881424 19309 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 11:42:44.949191 19298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:42:44.949519 19298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:42:44.949585 19298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:42:44.953866 19298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:42:44.953927 19298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:42:44.954008 19298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:42:44.959905 19298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.96.20:34403/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:35883
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=34809
--tserver_master_addrs=127.6.96.62:36855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:42:44.960991 19298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:42:44.962324 19298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:42:44.972056 19333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:44.972074 19336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:44.973062 19334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:44.973412 19298 server_base.cc:1034] running on GCE node
I20260926 11:42:44.973882 19298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:42:44.974650 19298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:42:44.975816 19298 hybrid_clock.cc:648] HybridClock initialized: now 1790422964975777 us; error 54 us; skew 500 ppm
Sep 26 11:42:44 dist-test-slave-xxxx krb5kdc[18263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422964, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:44.979400 19298 init.cc:377] Logged in from keytab as kudu/127.6.96.1@KRBTEST.COM (short username kudu)
I20260926 11:42:44.980602 19298 webserver.cc:460] Webserver started at http://127.6.96.1:34809/ using document root <none> and password file <none>
I20260926 11:42:44.981308 19298 fs_manager.cc:359] Metadata directory not provided
I20260926 11:42:44.981419 19298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 11:42:44 dist-test-slave-xxxx krb5kdc[18263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422964, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for HTTP/127.6.96.20@KRBTEST.COM
I20260926 11:42:45.011399 19298 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 11:42:45.014348 19344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:45.015635 19298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 11:42:45.015744 19298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "7a202cf1ada7405a9775e9265acc2c08"
format_stamp: "Formatted at 2026-09-26 11:42:43 on dist-test-slave-xxxx"
server_key: "227e6997360ed5fb9ef6cd164d4d0239"
server_key_iv: "8ea51b214477598da3c7de152a479a01"
server_key_version: "kuduclusterkey@0"
I20260926 11:42:45.016327 19298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:42:45.035830 19298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:42:45.038877 19298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:42:45.039062 19298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:42:45.039690 19298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:42:45.040603 19298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:42:45.040650 19298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:45.040701 19298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:42:45.040724 19298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:45.050664 19298 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:35883
I20260926 11:42:45.051502 19457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:35883 every 8 connection(s)
I20260926 11:42:45.052611 19298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:42:45.061262 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 19298
I20260926 11:42:45.061748 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.96.20:34403/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:35867
--local_ip_for_outbound_sockets=127.6.96.2
--tserver_master_addrs=127.6.96.62:36855
--webserver_port=38277
--webserver_interface=127.6.96.2
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.96.20:34403/kms with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 11:42:45 dist-test-slave-xxxx krb5kdc[18263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422964, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:42:45.079034 19460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:45.056237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:36855 (local address 127.6.96.1:51483)
0926 11:42:45.059160 (+ 2923us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:45.059224 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:45.060252 (+ 1028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:45.062408 (+ 2156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:45.062417 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:45.062753 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:42:45.063434 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:45.063454 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:45.065312 (+ 1858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:45.065316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:45.066734 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:45.066745 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:45.067980 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:45.067985 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:45.068179 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:45.070919 (+ 2740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:42:45.070941 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:42:45.072487 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:42:45.075231 (+ 2744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:45.075237 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:45.075252 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:45.075598 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:45.076006 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:45.076009 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:45.076011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:45.076197 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:45.077017 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:42:45.077024 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:42:45.077365 (+ 341us) client_negotiation.cc:769] Sending connection context
0926 11:42:45.077578 (+ 213us) client_negotiation.cc:240] Negotiation successful
0926 11:42:45.077940 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2686,"thread_start_us":123,"threads_started":1}
I20260926 11:42:45.080879 19461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:45.057641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:51483 (local address 127.6.96.62:36855)
0926 11:42:45.060762 (+ 3121us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:45.060784 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:45.060826 (+ 42us) server_negotiation.cc:407] Connection header received
0926 11:42:45.061114 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:45.061129 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:45.061555 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:45.061917 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:45.063643 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:45.065153 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:45.066897 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:45.067617 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:45.067765 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:45.072724 (+ 4959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:45.072731 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:45.072747 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:45.072782 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:45.075091 (+ 2309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:45.075737 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:45.075744 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:45.075750 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:45.075894 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:45.076305 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:45.076309 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:45.076311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:45.076827 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:45.076958 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:45.079611 (+ 2653us) server_negotiation.cc:299] Negotiation successful
0926 11:42:45.079909 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2967,"thread_start_us":116,"threads_started":1}
I20260926 11:42:45.083839 19458 heartbeater.cc:347] Connected to a master server at 127.6.96.62:36855
I20260926 11:42:45.084082 19458 heartbeater.cc:464] Registering TS with master...
I20260926 11:42:45.084656 19458 heartbeater.cc:511] Master 127.6.96.62:36855 requested a full tablet report, sending...
W20260926 11:42:45.086916 19458 heartbeater.cc:644] Failed to heartbeat to 127.6.96.62:36855 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 11:42:45.087040 19458 heartbeater.cc:464] Registering TS with master...
I20260926 11:42:45.087226 19458 heartbeater.cc:511] Master 127.6.96.62:36855 requested a full tablet report, sending...
I20260926 11:42:45.087993 19458 heartbeater.cc:464] Registering TS with master...
I20260926 11:42:45.088165 19458 heartbeater.cc:511] Master 127.6.96.62:36855 requested a full tablet report, sending...
W20260926 11:42:45.088855 19458 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 11:42:45.209764 19462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:42:45.210086 19462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:42:45.210155 19462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:42:45.214527 19462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:42:45.214593 19462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:42:45.214674 19462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:42:45.220140 19462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.96.20:34403/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:35867
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=38277
--tserver_master_addrs=127.6.96.62:36855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:42:45.221280 19462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:42:45.222365 19462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:42:45.233206 19468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:45.233232 19467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:45.233803 19462 server_base.cc:1034] running on GCE node
W20260926 11:42:45.234081 19470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:45.234613 19462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:42:45.235510 19462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:42:45.236727 19462 hybrid_clock.cc:648] HybridClock initialized: now 1790422965236697 us; error 43 us; skew 500 ppm
Sep 26 11:42:45 dist-test-slave-xxxx krb5kdc[18263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422965, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:45.240201 19462 init.cc:377] Logged in from keytab as kudu/127.6.96.2@KRBTEST.COM (short username kudu)
I20260926 11:42:45.241199 19462 webserver.cc:460] Webserver started at http://127.6.96.2:38277/ using document root <none> and password file <none>
I20260926 11:42:45.241755 19462 fs_manager.cc:359] Metadata directory not provided
I20260926 11:42:45.241837 19462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 11:42:45 dist-test-slave-xxxx krb5kdc[18263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422965, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for HTTP/127.6.96.20@KRBTEST.COM
I20260926 11:42:45.260402 19462 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.003s
I20260926 11:42:45.263520 19478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:45.265164 19462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:42:45.265300 19462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "08855d2e07084434b67250db07813e81"
format_stamp: "Formatted at 2026-09-26 11:42:44 on dist-test-slave-xxxx"
server_key: "70815587b211b291396dd756249542d5"
server_key_iv: "1611703744f8434aca482fcc32cfaf6f"
server_key_version: "kuduclusterkey@0"
I20260926 11:42:45.266052 19462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:42:45.283951 19462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:42:45.288054 19462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:42:45.288269 19462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:42:45.289017 19462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:42:45.290125 19462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:42:45.290198 19462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:45.290261 19462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:42:45.290295 19462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:45.300846 19462 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:35867
I20260926 11:42:45.300882 19591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:35867 every 8 connection(s)
I20260926 11:42:45.301872 19462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:42:45.309578 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 19462
I20260926 11:42:45.310103 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.96.20:34403/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:44035
--local_ip_for_outbound_sockets=127.6.96.3
--tserver_master_addrs=127.6.96.62:36855
--webserver_port=46103
--webserver_interface=127.6.96.3
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.96.20:34403/kms with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 11:42:45 dist-test-slave-xxxx krb5kdc[18263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422965, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:42:45.317899 19461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:45.305343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:41207 (local address 127.6.96.62:36855)
0926 11:42:45.305462 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:45.305466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:45.305483 (+ 17us) server_negotiation.cc:407] Connection header received
0926 11:42:45.305996 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:45.305999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:45.306056 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:45.306129 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:45.307504 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:45.308296 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:45.309889 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:45.310147 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:45.310244 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:45.313676 (+ 3432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:45.313680 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:45.313683 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:45.313715 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:45.315340 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:45.315986 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:45.315990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:45.315992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:45.316082 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:45.316498 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:45.316502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:45.316503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:45.316742 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:45.316814 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:45.317551 (+ 737us) server_negotiation.cc:299] Negotiation successful
0926 11:42:45.317695 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 11:42:45.319983 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:45.304307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:36855 (local address 127.6.96.2:41207)
0926 11:42:45.304789 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:45.304842 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:45.305780 (+ 938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:45.306325 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:45.306332 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:45.306614 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:42:45.307314 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:45.307334 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:45.308461 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:45.308464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:45.309772 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:45.309784 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:45.310432 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:45.310436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:45.310586 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:45.311749 (+ 1163us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:42:45.311771 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:42:45.313467 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:42:45.315478 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:45.315485 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:45.315503 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:45.315854 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:45.316198 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:45.316201 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:45.316203 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:45.316387 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:45.316894 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:42:45.316901 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:42:45.317266 (+ 365us) client_negotiation.cc:769] Sending connection context
0926 11:42:45.318465 (+ 1199us) client_negotiation.cc:240] Negotiation successful
0926 11:42:45.318867 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":113,"threads_started":1}
I20260926 11:42:45.321755 19592 heartbeater.cc:347] Connected to a master server at 127.6.96.62:36855
I20260926 11:42:45.322017 19592 heartbeater.cc:464] Registering TS with master...
I20260926 11:42:45.322579 19592 heartbeater.cc:511] Master 127.6.96.62:36855 requested a full tablet report, sending...
W20260926 11:42:45.324397 19592 heartbeater.cc:644] Failed to heartbeat to 127.6.96.62:36855 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 11:42:45.324527 19592 heartbeater.cc:464] Registering TS with master...
I20260926 11:42:45.324718 19592 heartbeater.cc:511] Master 127.6.96.62:36855 requested a full tablet report, sending...
I20260926 11:42:45.325538 19592 heartbeater.cc:464] Registering TS with master...
I20260926 11:42:45.325721 19592 heartbeater.cc:511] Master 127.6.96.62:36855 requested a full tablet report, sending...
W20260926 11:42:45.326534 19592 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 11:42:45.483649 19595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:42:45.484061 19595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:42:45.484138 19595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:42:45.490228 19595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:42:45.490319 19595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:42:45.490407 19595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:42:45.498490 19595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.96.20:34403/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:44035
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=46103
--tserver_master_addrs=127.6.96.62:36855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:42:45.500038 19595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:42:45.501413 19595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:42:45.513650 19602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:45.513640 19601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:45.513653 19604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:45.515116 19595 server_base.cc:1034] running on GCE node
I20260926 11:42:45.515738 19595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:42:45.516577 19595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:42:45.519868 19595 hybrid_clock.cc:648] HybridClock initialized: now 1790422965519793 us; error 91 us; skew 500 ppm
Sep 26 11:42:45 dist-test-slave-xxxx krb5kdc[18263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422965, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:45.523623 19595 init.cc:377] Logged in from keytab as kudu/127.6.96.3@KRBTEST.COM (short username kudu)
I20260926 11:42:45.524747 19595 webserver.cc:460] Webserver started at http://127.6.96.3:46103/ using document root <none> and password file <none>
I20260926 11:42:45.525491 19595 fs_manager.cc:359] Metadata directory not provided
I20260926 11:42:45.525616 19595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 11:42:45 dist-test-slave-xxxx krb5kdc[18263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422965, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for HTTP/127.6.96.20@KRBTEST.COM
I20260926 11:42:45.554701 19595 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.003s sys 0.000s
I20260926 11:42:45.557482 19612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:45.559165 19595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 11:42:45.559289 19595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "ec3eb9d834e5443a91c605891cb036b3"
format_stamp: "Formatted at 2026-09-26 11:42:44 on dist-test-slave-xxxx"
server_key: "388286b6776fbcbc12af94ee9bed2292"
server_key_iv: "0aa7df5dce08c2911257e50b37e044bf"
server_key_version: "kuduclusterkey@0"
I20260926 11:42:45.559885 19595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:42:45.571831 19595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:42:45.575227 19595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:42:45.575404 19595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:42:45.575989 19595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:42:45.576838 19595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:42:45.576884 19595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:45.576936 19595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:42:45.577003 19595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:45.586390 19595 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:44035
I20260926 11:42:45.587831 19595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:42:45.588673 19725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:44035 every 8 connection(s)
I20260926 11:42:45.589444 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 19595
Sep 26 11:42:45 dist-test-slave-xxxx krb5kdc[18263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422965, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:42:45.602622 19461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:45.590505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46622 (local address 127.6.96.62:36855)
0926 11:42:45.592431 (+ 1926us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:45.592435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:45.592451 (+ 16us) server_negotiation.cc:407] Connection header received
0926 11:42:45.592507 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:45.592510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:45.592557 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:45.592638 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:45.596110 (+ 3472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:45.596861 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:45.598111 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:45.598370 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:45.598452 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:45.599859 (+ 1407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:45.599863 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:45.599865 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:45.599889 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:45.601107 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:45.601475 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:45.601479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:45.601481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:45.601557 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:45.601788 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:45.601792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:45.601793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:45.601955 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:45.602151 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:45.602287 (+ 136us) server_negotiation.cc:299] Negotiation successful
0926 11:42:45.602496 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1839}
I20260926 11:42:45.606698 19730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:45.592841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:42439 (local address 127.6.96.62:36855)
0926 11:42:45.593055 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:45.593059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:45.593073 (+ 14us) server_negotiation.cc:407] Connection header received
0926 11:42:45.593621 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:45.593623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:45.593664 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:45.593750 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:45.595120 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:45.595829 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:45.597377 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:45.597615 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:45.597697 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:45.603204 (+ 5507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:45.603207 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:45.603210 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:45.603232 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:45.604825 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:45.605291 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:45.605294 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:45.605296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:45.605365 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:45.605683 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:45.605687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:45.605688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:45.605843 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:45.605920 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:45.606426 (+ 506us) server_negotiation.cc:299] Negotiation successful
0926 11:42:45.606538 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":89,"threads_started":1}
I20260926 11:42:45.607543 19729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:45.591422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:36855 (local address 127.6.96.3:42439)
0926 11:42:45.591939 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:45.591997 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:45.593330 (+ 1333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:45.594021 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:45.594046 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:45.594312 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:42:45.594943 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:45.594965 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:45.595948 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:45.595952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:45.597279 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:45.597291 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:45.597750 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:45.597754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:45.597929 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:45.598527 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:42:45.598548 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:42:45.603052 (+ 4504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:42:45.604928 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:45.604935 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:45.604951 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:45.605194 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:45.605438 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:45.605442 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:45.605444 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:45.605590 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:45.605933 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:42:45.605940 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:42:45.606198 (+ 258us) client_negotiation.cc:769] Sending connection context
0926 11:42:45.606358 (+ 160us) client_negotiation.cc:240] Negotiation successful
0926 11:42:45.606686 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":134,"threads_started":1}
I20260926 11:42:45.608557 19726 heartbeater.cc:347] Connected to a master server at 127.6.96.62:36855
I20260926 11:42:45.608803 19726 heartbeater.cc:464] Registering TS with master...
I20260926 11:42:45.609405 19726 heartbeater.cc:511] Master 127.6.96.62:36855 requested a full tablet report, sending...
W20260926 11:42:45.611084 19726 heartbeater.cc:644] Failed to heartbeat to 127.6.96.62:36855 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 11:42:45.611203 19726 heartbeater.cc:464] Registering TS with master...
I20260926 11:42:45.611366 19726 heartbeater.cc:511] Master 127.6.96.62:36855 requested a full tablet report, sending...
I20260926 11:42:45.612092 19726 heartbeater.cc:464] Registering TS with master...
I20260926 11:42:45.612248 19726 heartbeater.cc:511] Master 127.6.96.62:36855 requested a full tablet report, sending...
W20260926 11:42:45.612927 19726 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 26 11:42:45 dist-test-slave-xxxx krb5kdc[18263](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790422965, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:46.089087 19458 heartbeater.cc:464] Registering TS with master...
I20260926 11:42:46.089423 19458 heartbeater.cc:511] Master 127.6.96.62:36855 requested a full tablet report, sending...
I20260926 11:42:46.090641 19260 ts_manager.cc:194] Registered new tserver with Master: 7a202cf1ada7405a9775e9265acc2c08 (127.6.96.1:35883)
I20260926 11:42:46.092160 19260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.1@KRBTEST.COM'} at 127.6.96.1:51483
I20260926 11:42:46.326766 19592 heartbeater.cc:464] Registering TS with master...
I20260926 11:42:46.327108 19592 heartbeater.cc:511] Master 127.6.96.62:36855 requested a full tablet report, sending...
I20260926 11:42:46.328003 19260 ts_manager.cc:194] Registered new tserver with Master: 08855d2e07084434b67250db07813e81 (127.6.96.2:35867)
I20260926 11:42:46.328823 19260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.2@KRBTEST.COM'} at 127.6.96.2:41207
Sep 26 11:42:46 dist-test-slave-xxxx krb5kdc[18263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422965, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for HTTP/127.6.96.20@KRBTEST.COM
I20260926 11:42:46.613145 19726 heartbeater.cc:464] Registering TS with master...
I20260926 11:42:46.613444 19726 heartbeater.cc:511] Master 127.6.96.62:36855 requested a full tablet report, sending...
I20260926 11:42:46.614151 19260 ts_manager.cc:194] Registered new tserver with Master: ec3eb9d834e5443a91c605891cb036b3 (127.6.96.3:44035)
I20260926 11:42:46.615059 19260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.3@KRBTEST.COM'} at 127.6.96.3:42439
Sep 26 11:42:46 dist-test-slave-xxxx krb5kdc[18263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422965, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for HTTP/127.6.96.20@KRBTEST.COM
I20260926 11:42:46.624415 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 11:42:46.641656 19748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:46.627496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51000 (local address 127.6.96.62:36855)
0926 11:42:46.628260 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:46.628269 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:46.629285 (+ 1016us) server_negotiation.cc:407] Connection header received
0926 11:42:46.630948 (+ 1663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:46.630952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:46.631014 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:46.631081 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:46.632066 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:46.632910 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:46.634472 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:46.634768 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:46.634854 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:46.637332 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:46.637336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:46.637339 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:46.637372 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:46.639054 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:46.639728 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:46.639732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:46.639734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:46.639835 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:46.640418 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:46.640421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:46.640423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:46.640642 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:46.640725 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:46.641369 (+ 644us) server_negotiation.cc:299] Negotiation successful
0926 11:42:46.641512 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":84,"threads_started":1}
I20260926 11:42:46.646376 19260 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51000:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 11:42:46.651750 19260 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:42:46.691490 19755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:46.669016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:35867 (local address 127.0.0.1:56910)
0926 11:42:46.677044 (+ 8028us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:46.677134 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:46.677299 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:46.686905 (+ 9606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:46.686909 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:46.686942 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:42:46.687054 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:46.687061 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:46.688430 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:46.688433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:46.689469 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:46.689482 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:46.690879 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:46.690883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:46.691042 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:46.691088 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 11:42:46.691200 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 11:42:46.691325 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7712,"thread_start_us":118,"threads_started":1}
I20260926 11:42:46.692215 19757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:46.669359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:35883 (local address 127.0.0.1:32814)
0926 11:42:46.679079 (+ 9720us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:46.679095 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:46.679208 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:46.682658 (+ 3450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:46.682661 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:46.682681 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:42:46.682786 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:46.682793 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:46.685306 (+ 2513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:46.685309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:46.686733 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:46.686746 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:46.691811 (+ 5065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:46.691815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:46.691923 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:46.691949 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 11:42:46.692010 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 11:42:46.692083 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9638,"thread_start_us":103,"threads_started":1}
I20260926 11:42:46.692619 19756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:46.669146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56910 (local address 127.6.96.2:35867)
0926 11:42:46.672977 (+ 3831us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:46.672984 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:46.679961 (+ 6977us) server_negotiation.cc:407] Connection header received
0926 11:42:46.680055 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:46.680061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:46.680414 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:46.680582 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:42:46.687223 (+ 6641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:46.688314 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:46.689625 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:46.690508 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:46.690614 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:46.690790 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:46.692265 (+ 1475us) server_negotiation.cc:299] Negotiation successful
0926 11:42:46.692483 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3708,"thread_start_us":91,"threads_started":1}
I20260926 11:42:46.694528 19759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:46.670004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32814 (local address 127.6.96.1:35883)
0926 11:42:46.670310 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:46.670317 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:46.679335 (+ 9018us) server_negotiation.cc:407] Connection header received
0926 11:42:46.679461 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:46.679469 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:46.679751 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:46.679925 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:42:46.682962 (+ 3037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:46.684277 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:46.689066 (+ 4789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:46.691025 (+ 1959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:46.691138 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:46.691287 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:46.694249 (+ 2962us) server_negotiation.cc:299] Negotiation successful
0926 11:42:46.694373 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":66,"threads_started":1}
I20260926 11:42:46.696488 19523 tablet_service.cc:1467] Processing CreateTablet for tablet e96be8b253db4a5e8e54f3df2bf79a62 (DEFAULT_TABLE table=test-table [id=ce0132b4ab634946bf8ff257cd889d5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:42:46.697688 19523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e96be8b253db4a5e8e54f3df2bf79a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:42:46.697623 19389 tablet_service.cc:1467] Processing CreateTablet for tablet e96be8b253db4a5e8e54f3df2bf79a62 (DEFAULT_TABLE table=test-table [id=ce0132b4ab634946bf8ff257cd889d5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:42:46.698699 19389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e96be8b253db4a5e8e54f3df2bf79a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:42:46.704975 19758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:46.669814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:44035 (local address 127.0.0.1:44892)
0926 11:42:46.681335 (+ 11521us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:46.681351 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:46.681551 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:46.682303 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:46.682310 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:46.682337 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:42:46.682528 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:46.682537 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:46.688289 (+ 5752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:46.688292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:46.691366 (+ 3074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:46.691375 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:46.704486 (+ 13111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:46.704491 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:46.704607 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:46.704642 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 11:42:46.704702 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 11:42:46.704801 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11444,"thread_start_us":82,"threads_started":1}
I20260926 11:42:46.705900 19760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:46.670323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44892 (local address 127.6.96.3:44035)
0926 11:42:46.670638 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:46.670645 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:46.681464 (+ 10819us) server_negotiation.cc:407] Connection header received
0926 11:42:46.681716 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:46.681723 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:46.681930 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:46.682063 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:42:46.686947 (+ 4884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:46.688141 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:46.702920 (+ 14779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:46.703807 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:46.705386 (+ 1579us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:46.705578 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:46.705645 (+ 67us) server_negotiation.cc:299] Negotiation successful
0926 11:42:46.705757 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":129,"threads_started":1}
I20260926 11:42:46.716320 19761 tablet_bootstrap.cc:492] T e96be8b253db4a5e8e54f3df2bf79a62 P 08855d2e07084434b67250db07813e81: Bootstrap starting.
I20260926 11:42:46.716039 19660 tablet_service.cc:1467] Processing CreateTablet for tablet e96be8b253db4a5e8e54f3df2bf79a62 (DEFAULT_TABLE table=test-table [id=ce0132b4ab634946bf8ff257cd889d5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:42:46.717370 19660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e96be8b253db4a5e8e54f3df2bf79a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:42:46.718921 19761 tablet_bootstrap.cc:654] T e96be8b253db4a5e8e54f3df2bf79a62 P 08855d2e07084434b67250db07813e81: Neither blocks nor log segments found. Creating new log.
I20260926 11:42:46.719846 19761 log.cc:824] T e96be8b253db4a5e8e54f3df2bf79a62 P 08855d2e07084434b67250db07813e81: Log is configured to *not* fsync() on all Append() calls
I20260926 11:42:46.728942 19764 tablet_bootstrap.cc:492] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3: Bootstrap starting.
I20260926 11:42:46.729513 19762 tablet_bootstrap.cc:492] T e96be8b253db4a5e8e54f3df2bf79a62 P 7a202cf1ada7405a9775e9265acc2c08: Bootstrap starting.
I20260926 11:42:46.730206 19761 tablet_bootstrap.cc:492] T e96be8b253db4a5e8e54f3df2bf79a62 P 08855d2e07084434b67250db07813e81: No bootstrap required, opened a new log
I20260926 11:42:46.730504 19761 ts_tablet_manager.cc:1397] T e96be8b253db4a5e8e54f3df2bf79a62 P 08855d2e07084434b67250db07813e81: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.005s
I20260926 11:42:46.732107 19764 tablet_bootstrap.cc:654] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3: Neither blocks nor log segments found. Creating new log.
I20260926 11:42:46.734472 19762 tablet_bootstrap.cc:654] T e96be8b253db4a5e8e54f3df2bf79a62 P 7a202cf1ada7405a9775e9265acc2c08: Neither blocks nor log segments found. Creating new log.
I20260926 11:42:46.734716 19761 raft_consensus.cc:348] T e96be8b253db4a5e8e54f3df2bf79a62 P 08855d2e07084434b67250db07813e81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08855d2e07084434b67250db07813e81" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35867 } } peers { permanent_uuid: "ec3eb9d834e5443a91c605891cb036b3" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 44035 } } peers { permanent_uuid: "7a202cf1ada7405a9775e9265acc2c08" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 35883 } }
I20260926 11:42:46.735146 19761 raft_consensus.cc:374] T e96be8b253db4a5e8e54f3df2bf79a62 P 08855d2e07084434b67250db07813e81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:42:46.735252 19761 raft_consensus.cc:729] T e96be8b253db4a5e8e54f3df2bf79a62 P 08855d2e07084434b67250db07813e81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08855d2e07084434b67250db07813e81, State: Initialized, Role: FOLLOWER
I20260926 11:42:46.735482 19764 log.cc:824] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3: Log is configured to *not* fsync() on all Append() calls
I20260926 11:42:46.736783 19762 log.cc:824] T e96be8b253db4a5e8e54f3df2bf79a62 P 7a202cf1ada7405a9775e9265acc2c08: Log is configured to *not* fsync() on all Append() calls
I20260926 11:42:46.737931 19764 tablet_bootstrap.cc:492] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3: No bootstrap required, opened a new log
I20260926 11:42:46.738134 19764 ts_tablet_manager.cc:1397] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260926 11:42:46.742290 19764 raft_consensus.cc:348] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08855d2e07084434b67250db07813e81" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35867 } } peers { permanent_uuid: "ec3eb9d834e5443a91c605891cb036b3" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 44035 } } peers { permanent_uuid: "7a202cf1ada7405a9775e9265acc2c08" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 35883 } }
I20260926 11:42:46.742584 19764 raft_consensus.cc:374] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:42:46.742632 19764 raft_consensus.cc:729] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3eb9d834e5443a91c605891cb036b3, State: Initialized, Role: FOLLOWER
I20260926 11:42:46.743283 19764 consensus_queue.cc:260] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08855d2e07084434b67250db07813e81" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35867 } } peers { permanent_uuid: "ec3eb9d834e5443a91c605891cb036b3" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 44035 } } peers { permanent_uuid: "7a202cf1ada7405a9775e9265acc2c08" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 35883 } }
I20260926 11:42:46.744544 19764 ts_tablet_manager.cc:1428] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260926 11:42:46.745018 19726 heartbeater.cc:503] Master 127.6.96.62:36855 was elected leader, sending a full tablet report...
I20260926 11:42:46.747010 19762 tablet_bootstrap.cc:492] T e96be8b253db4a5e8e54f3df2bf79a62 P 7a202cf1ada7405a9775e9265acc2c08: No bootstrap required, opened a new log
I20260926 11:42:46.747232 19762 ts_tablet_manager.cc:1397] T e96be8b253db4a5e8e54f3df2bf79a62 P 7a202cf1ada7405a9775e9265acc2c08: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.000s
I20260926 11:42:46.751829 19762 raft_consensus.cc:348] T e96be8b253db4a5e8e54f3df2bf79a62 P 7a202cf1ada7405a9775e9265acc2c08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08855d2e07084434b67250db07813e81" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35867 } } peers { permanent_uuid: "ec3eb9d834e5443a91c605891cb036b3" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 44035 } } peers { permanent_uuid: "7a202cf1ada7405a9775e9265acc2c08" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 35883 } }
I20260926 11:42:46.752146 19762 raft_consensus.cc:374] T e96be8b253db4a5e8e54f3df2bf79a62 P 7a202cf1ada7405a9775e9265acc2c08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:42:46.752204 19762 raft_consensus.cc:729] T e96be8b253db4a5e8e54f3df2bf79a62 P 7a202cf1ada7405a9775e9265acc2c08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a202cf1ada7405a9775e9265acc2c08, State: Initialized, Role: FOLLOWER
I20260926 11:42:46.752791 19762 consensus_queue.cc:260] T e96be8b253db4a5e8e54f3df2bf79a62 P 7a202cf1ada7405a9775e9265acc2c08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08855d2e07084434b67250db07813e81" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35867 } } peers { permanent_uuid: "ec3eb9d834e5443a91c605891cb036b3" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 44035 } } peers { permanent_uuid: "7a202cf1ada7405a9775e9265acc2c08" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 35883 } }
I20260926 11:42:46.754006 19762 ts_tablet_manager.cc:1428] T e96be8b253db4a5e8e54f3df2bf79a62 P 7a202cf1ada7405a9775e9265acc2c08: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260926 11:42:46.754413 19458 heartbeater.cc:503] Master 127.6.96.62:36855 was elected leader, sending a full tablet report...
I20260926 11:42:46.759305 19761 consensus_queue.cc:260] T e96be8b253db4a5e8e54f3df2bf79a62 P 08855d2e07084434b67250db07813e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08855d2e07084434b67250db07813e81" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35867 } } peers { permanent_uuid: "ec3eb9d834e5443a91c605891cb036b3" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 44035 } } peers { permanent_uuid: "7a202cf1ada7405a9775e9265acc2c08" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 35883 } }
I20260926 11:42:46.760690 19761 ts_tablet_manager.cc:1428] T e96be8b253db4a5e8e54f3df2bf79a62 P 08855d2e07084434b67250db07813e81: Time spent starting tablet: real 0.030s user 0.003s sys 0.002s
I20260926 11:42:46.761104 19592 heartbeater.cc:503] Master 127.6.96.62:36855 was elected leader, sending a full tablet report...
W20260926 11:42:46.804759 19593 tablet.cc:2369] T e96be8b253db4a5e8e54f3df2bf79a62 P 08855d2e07084434b67250db07813e81: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 11:42:46.804823 19459 tablet.cc:2369] T e96be8b253db4a5e8e54f3df2bf79a62 P 7a202cf1ada7405a9775e9265acc2c08: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 11:42:46.840138 19727 tablet.cc:2369] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:42:46.880118 19767 raft_consensus.cc:482] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:42:46.880362 19767 raft_consensus.cc:504] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08855d2e07084434b67250db07813e81" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35867 } } peers { permanent_uuid: "ec3eb9d834e5443a91c605891cb036b3" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 44035 } } peers { permanent_uuid: "7a202cf1ada7405a9775e9265acc2c08" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 35883 } }
I20260926 11:42:46.881734 19767 leader_election.cc:288] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08855d2e07084434b67250db07813e81 (127.6.96.2:35867), 7a202cf1ada7405a9775e9265acc2c08 (127.6.96.1:35883)
I20260926 11:42:46.887439 19759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:46.882629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:36993 (local address 127.6.96.1:35883)
0926 11:42:46.882742 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:46.882746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:46.882987 (+ 241us) server_negotiation.cc:407] Connection header received
0926 11:42:46.883130 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:46.883133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:46.883189 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:46.883268 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:42:46.883725 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:46.884511 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:46.886149 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:46.886731 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:46.886814 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:46.886906 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:46.887217 (+ 311us) server_negotiation.cc:299] Negotiation successful
0926 11:42:46.887294 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 11:42:46.887492 19756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:46.882333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:39395 (local address 127.6.96.2:35867)
0926 11:42:46.882476 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:46.882481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:46.882718 (+ 237us) server_negotiation.cc:407] Connection header received
0926 11:42:46.883245 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:46.883248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:46.883297 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:46.883343 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:42:46.883796 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:46.884587 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:46.886158 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:46.886762 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:46.886874 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:46.886938 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:46.887269 (+ 331us) server_negotiation.cc:299] Negotiation successful
0926 11:42:46.887349 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 11:42:46.887930 19770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:46.881997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:35867 (local address 127.6.96.3:39395)
0926 11:42:46.882629 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:46.882643 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:46.883129 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:46.883484 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:46.883487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:46.883504 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:42:46.883667 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:46.883673 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:46.884730 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:46.884733 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:46.886017 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:46.886028 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:46.887031 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:46.887034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:46.887140 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:46.887161 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 11:42:46.887692 (+ 531us) client_negotiation.cc:240] Negotiation successful
0926 11:42:46.887772 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":29,"thread_start_us":106,"threads_started":1}
I20260926 11:42:46.888134 19771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:46.882228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:35883 (local address 127.6.96.3:36993)
0926 11:42:46.882904 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:46.882916 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:46.883027 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:46.883382 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:46.883385 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:46.883403 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:42:46.883599 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:46.883606 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:46.884648 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:46.884651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:46.886012 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:46.886023 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:46.886995 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:46.886998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:46.887094 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:46.887113 (+ 19us) client_negotiation.cc:769] Sending connection context
0926 11:42:46.887910 (+ 797us) client_negotiation.cc:240] Negotiation successful
0926 11:42:46.887989 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"mutex_wait_us":73,"thread_start_us":106,"threads_started":1}
I20260926 11:42:46.888789 19527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e96be8b253db4a5e8e54f3df2bf79a62" candidate_uuid: "ec3eb9d834e5443a91c605891cb036b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08855d2e07084434b67250db07813e81" is_pre_election: true
I20260926 11:42:46.888935 19394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e96be8b253db4a5e8e54f3df2bf79a62" candidate_uuid: "ec3eb9d834e5443a91c605891cb036b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a202cf1ada7405a9775e9265acc2c08" is_pre_election: true
I20260926 11:42:46.889128 19527 raft_consensus.cc:2454] T e96be8b253db4a5e8e54f3df2bf79a62 P 08855d2e07084434b67250db07813e81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec3eb9d834e5443a91c605891cb036b3 in term 0.
I20260926 11:42:46.889235 19394 raft_consensus.cc:2454] T e96be8b253db4a5e8e54f3df2bf79a62 P 7a202cf1ada7405a9775e9265acc2c08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec3eb9d834e5443a91c605891cb036b3 in term 0.
I20260926 11:42:46.889755 19613 leader_election.cc:302] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a202cf1ada7405a9775e9265acc2c08, ec3eb9d834e5443a91c605891cb036b3; no voters:
I20260926 11:42:46.890074 19767 raft_consensus.cc:2781] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:42:46.890147 19767 raft_consensus.cc:482] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:42:46.890185 19767 raft_consensus.cc:3037] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:42:46.891506 19767 raft_consensus.cc:504] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08855d2e07084434b67250db07813e81" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35867 } } peers { permanent_uuid: "ec3eb9d834e5443a91c605891cb036b3" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 44035 } } peers { permanent_uuid: "7a202cf1ada7405a9775e9265acc2c08" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 35883 } }
I20260926 11:42:46.891870 19767 leader_election.cc:288] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3 [CANDIDATE]: Term 1 election: Requested vote from peers 08855d2e07084434b67250db07813e81 (127.6.96.2:35867), 7a202cf1ada7405a9775e9265acc2c08 (127.6.96.1:35883)
I20260926 11:42:46.892275 19394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e96be8b253db4a5e8e54f3df2bf79a62" candidate_uuid: "ec3eb9d834e5443a91c605891cb036b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a202cf1ada7405a9775e9265acc2c08"
I20260926 11:42:46.892328 19527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e96be8b253db4a5e8e54f3df2bf79a62" candidate_uuid: "ec3eb9d834e5443a91c605891cb036b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08855d2e07084434b67250db07813e81"
I20260926 11:42:46.892395 19394 raft_consensus.cc:3037] T e96be8b253db4a5e8e54f3df2bf79a62 P 7a202cf1ada7405a9775e9265acc2c08 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:42:46.892462 19527 raft_consensus.cc:3037] T e96be8b253db4a5e8e54f3df2bf79a62 P 08855d2e07084434b67250db07813e81 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:42:46.893738 19394 raft_consensus.cc:2454] T e96be8b253db4a5e8e54f3df2bf79a62 P 7a202cf1ada7405a9775e9265acc2c08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec3eb9d834e5443a91c605891cb036b3 in term 1.
I20260926 11:42:46.893862 19527 raft_consensus.cc:2454] T e96be8b253db4a5e8e54f3df2bf79a62 P 08855d2e07084434b67250db07813e81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec3eb9d834e5443a91c605891cb036b3 in term 1.
I20260926 11:42:46.894215 19613 leader_election.cc:302] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a202cf1ada7405a9775e9265acc2c08, ec3eb9d834e5443a91c605891cb036b3; no voters:
I20260926 11:42:46.894435 19767 raft_consensus.cc:2781] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:42:46.894661 19767 raft_consensus.cc:686] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3 [term 1 LEADER]: Becoming Leader. State: Replica: ec3eb9d834e5443a91c605891cb036b3, State: Running, Role: LEADER
I20260926 11:42:46.895056 19767 consensus_queue.cc:237] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08855d2e07084434b67250db07813e81" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35867 } } peers { permanent_uuid: "ec3eb9d834e5443a91c605891cb036b3" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 44035 } } peers { permanent_uuid: "7a202cf1ada7405a9775e9265acc2c08" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 35883 } }
I20260926 11:42:46.898695 19260 catalog_manager.cc:5495] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec3eb9d834e5443a91c605891cb036b3 (127.6.96.3). New cstate: current_term: 1 leader_uuid: "ec3eb9d834e5443a91c605891cb036b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08855d2e07084434b67250db07813e81" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec3eb9d834e5443a91c605891cb036b3" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 44035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a202cf1ada7405a9775e9265acc2c08" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 35883 } health_report { overall_health: UNKNOWN } } }
I20260926 11:42:46.915798 19760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:46.911737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44896 (local address 127.6.96.3:44035)
0926 11:42:46.911880 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:46.911884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:46.911901 (+ 17us) server_negotiation.cc:407] Connection header received
0926 11:42:46.911989 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:46.911992 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:46.912059 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:46.912170 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:42:46.912463 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:46.913256 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:46.914246 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:46.914473 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:46.914562 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:46.914881 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:42:46.915392 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:42:46.915512 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:46.915581 (+ 69us) server_negotiation.cc:299] Negotiation successful
0926 11:42:46.915641 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 11:42:46.921470 19527 raft_consensus.cc:1261] T e96be8b253db4a5e8e54f3df2bf79a62 P 08855d2e07084434b67250db07813e81 [term 1 FOLLOWER]: Refusing update from remote peer ec3eb9d834e5443a91c605891cb036b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:42:46.921502 19394 raft_consensus.cc:1261] T e96be8b253db4a5e8e54f3df2bf79a62 P 7a202cf1ada7405a9775e9265acc2c08 [term 1 FOLLOWER]: Refusing update from remote peer ec3eb9d834e5443a91c605891cb036b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:42:46.922168 19767 consensus_queue.cc:1035] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a202cf1ada7405a9775e9265acc2c08" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 35883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:42:46.922367 19772 consensus_queue.cc:1035] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08855d2e07084434b67250db07813e81" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:42:46.931219 19777 mvcc.cc:205] Tried to move back new op lower bound from 7333572472503349248 to 7333572472405303296. Current Snapshot: MvccSnapshot[applied={T|T < 7333572472503349248}]
I20260926 11:42:46.933612 19781 mvcc.cc:205] Tried to move back new op lower bound from 7333572472503349248 to 7333572472405303296. Current Snapshot: MvccSnapshot[applied={T|T < 7333572472503349248}]
I20260926 11:42:46.933627 19782 mvcc.cc:205] Tried to move back new op lower bound from 7333572472503349248 to 7333572472405303296. Current Snapshot: MvccSnapshot[applied={T|T < 7333572472503349248}]
I20260926 11:42:46.943511 19259 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:42:46.943701 19259 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:42:46.946280 19259 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bf07eb7b624846cb9d9a677c82057a12: Sending DeleteTablet for 3 replicas of tablet e96be8b253db4a5e8e54f3df2bf79a62
I20260926 11:42:46.947141 19389 tablet_service.cc:1514] Processing DeleteTablet for tablet e96be8b253db4a5e8e54f3df2bf79a62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:42:46 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:32814
I20260926 11:42:46.947165 19523 tablet_service.cc:1514] Processing DeleteTablet for tablet e96be8b253db4a5e8e54f3df2bf79a62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:42:46 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:56910
I20260926 11:42:46.947360 19660 tablet_service.cc:1514] Processing DeleteTablet for tablet e96be8b253db4a5e8e54f3df2bf79a62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:42:46 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:44892
I20260926 11:42:46.948041 19791 tablet_replica.cc:330] T e96be8b253db4a5e8e54f3df2bf79a62 P 7a202cf1ada7405a9775e9265acc2c08: stopping tablet replica
I20260926 11:42:46.948380 19791 raft_consensus.cc:2229] T e96be8b253db4a5e8e54f3df2bf79a62 P 7a202cf1ada7405a9775e9265acc2c08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:42:46.948725 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 19298
I20260926 11:42:46.948724 19791 raft_consensus.cc:2258] T e96be8b253db4a5e8e54f3df2bf79a62 P 7a202cf1ada7405a9775e9265acc2c08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:42:46.949069 19792 tablet_replica.cc:330] T e96be8b253db4a5e8e54f3df2bf79a62 P 08855d2e07084434b67250db07813e81: stopping tablet replica
I20260926 11:42:46.949380 19792 raft_consensus.cc:2229] T e96be8b253db4a5e8e54f3df2bf79a62 P 08855d2e07084434b67250db07813e81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:42:46.949684 19792 raft_consensus.cc:2258] T e96be8b253db4a5e8e54f3df2bf79a62 P 08855d2e07084434b67250db07813e81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:42:46.950060 19793 tablet_replica.cc:330] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3: stopping tablet replica
I20260926 11:42:46.950330 19793 raft_consensus.cc:2229] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3 [term 1 LEADER]: Raft consensus shutting down.
I20260926 11:42:46.950723 19793 raft_consensus.cc:2258] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:42:46.953049 19793 ts_tablet_manager.cc:1905] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:42:46.953349 19792 ts_tablet_manager.cc:1905] T e96be8b253db4a5e8e54f3df2bf79a62 P 08855d2e07084434b67250db07813e81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:42:46.956180 19792 ts_tablet_manager.cc:1918] T e96be8b253db4a5e8e54f3df2bf79a62 P 08855d2e07084434b67250db07813e81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260926 11:42:46.958170 19244 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.96.1:35883 (error 108)
I20260926 11:42:46.958900 19793 ts_tablet_manager.cc:1918] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:42:46.959033 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 19462
I20260926 11:42:46.959110 19792 log.cc:1196] T e96be8b253db4a5e8e54f3df2bf79a62 P 08855d2e07084434b67250db07813e81: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-1/wal/wals/e96be8b253db4a5e8e54f3df2bf79a62
I20260926 11:42:46.959383 19792 ts_tablet_manager.cc:1939] T e96be8b253db4a5e8e54f3df2bf79a62 P 08855d2e07084434b67250db07813e81: Deleting consensus metadata
I20260926 11:42:46.959568 19793 log.cc:1196] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/e96be8b253db4a5e8e54f3df2bf79a62
I20260926 11:42:46.959841 19793 ts_tablet_manager.cc:1939] T e96be8b253db4a5e8e54f3df2bf79a62 P ec3eb9d834e5443a91c605891cb036b3: Deleting consensus metadata
I20260926 11:42:46.961915 19758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:46.961583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:35883 (local address 127.0.0.1:32826)
0926 11:42:46.961758 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:46.961824 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.1:35883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260926 11:42:46.962141 19244 catalog_manager.cc:4584] TS 7a202cf1ada7405a9775e9265acc2c08 (127.6.96.1:35883): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e96be8b253db4a5e8e54f3df2bf79a62: Network error: Client connection negotiation failed: client connection to 127.6.96.1:35883: connect: Connection refused (error 111)
I20260926 11:42:46.963116 19245 catalog_manager.cc:4841] TS ec3eb9d834e5443a91c605891cb036b3 (127.6.96.3:44035): tablet e96be8b253db4a5e8e54f3df2bf79a62 (table test-table [id=ce0132b4ab634946bf8ff257cd889d5c]) successfully deleted
I20260926 11:42:46.966007 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 19595
I20260926 11:42:46.966106 19758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:46.965857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:35867 (local address 127.0.0.1:56924)
0926 11:42:46.965977 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:46.966029 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.2:35867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260926 11:42:46.970964 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 19227
I20260926 11:42:46.976392 6528 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 11:42:47.310762 6528 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 11:42:47.310950 6528 mini_ranger.cc:66] Stopping Ranger...
I20260926 11:42:47.645440 6528 mini_ranger.cc:68] Stopped Ranger
2026-09-26 11:42:47.645 UTC [18293] LOG: received smart shutdown request
2026-09-26 11:42:47.649 UTC [18293] LOG: background worker "logical replication launcher" (PID 18302) exited with exit code 1
2026-09-26 11:42:47.649 UTC [18297] LOG: shutting down
2026-09-26 11:42:47.804 UTC [18293] LOG: database system is shut down
2026-09-26T11:42:47Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47258 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:42:47 dist-test-slave-xxxx krb5kdc[19803](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:42:47 dist-test-slave-xxxx krb5kdc[19803](info): set up 2 sockets
Sep 26 11:42:47 dist-test-slave-xxxx krb5kdc[19803](info): commencing operation
krb5kdc: starting...
W20260926 11:42:48.858547 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:42:48 dist-test-slave-xxxx krb5kdc[19803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:42:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:42:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:42:48.974028 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:43579
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:36301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:43579,127.6.96.61:35373,127.6.96.60:43653
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:42:49.080077 19816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:42:49.080333 19816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:42:49.080379 19816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:42:49.084327 19816 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:42:49.084381 19816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:42:49.084398 19816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:42:49.084415 19816 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:42:49.084442 19816 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:42:49.089603 19816 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:36301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:43579,127.6.96.61:35373,127.6.96.60:43653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:43579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:42:49.090624 19816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:42:49.091619 19816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:42:49.099701 19821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:49.099701 19824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:49.099715 19822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:49.099951 19816 server_base.cc:1034] running on GCE node
I20260926 11:42:49.100636 19816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:42:49.101689 19816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:42:49.102864 19816 hybrid_clock.cc:648] HybridClock initialized: now 1790422969102844 us; error 30 us; skew 500 ppm
Sep 26 11:42:49 dist-test-slave-xxxx krb5kdc[19803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422969, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:49.105422 19816 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:42:49.106244 19816 webserver.cc:460] Webserver started at http://127.6.96.62:33753/ using document root <none> and password file <none>
I20260926 11:42:49.106750 19816 fs_manager.cc:359] Metadata directory not provided
I20260926 11:42:49.106804 19816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:42:49.107007 19816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:42:49.108927 19816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "ec5a4c708307474a82f00458ac47dbb4"
format_stamp: "Formatted at 2026-09-26 11:42:49 on dist-test-slave-xxxx"
server_key: "c7848ff2780e06475fc2bc33b56c9045"
server_key_iv: "24c6244cde0d010e24e88ec7bcc6fbc9"
server_key_version: "encryptionkey@0"
I20260926 11:42:49.109432 19816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "ec5a4c708307474a82f00458ac47dbb4"
format_stamp: "Formatted at 2026-09-26 11:42:49 on dist-test-slave-xxxx"
server_key: "c7848ff2780e06475fc2bc33b56c9045"
server_key_iv: "24c6244cde0d010e24e88ec7bcc6fbc9"
server_key_version: "encryptionkey@0"
I20260926 11:42:49.112879 19816 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260926 11:42:49.115114 19831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:49.116384 19816 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 11:42:49.116503 19816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "ec5a4c708307474a82f00458ac47dbb4"
format_stamp: "Formatted at 2026-09-26 11:42:49 on dist-test-slave-xxxx"
server_key: "c7848ff2780e06475fc2bc33b56c9045"
server_key_iv: "24c6244cde0d010e24e88ec7bcc6fbc9"
server_key_version: "encryptionkey@0"
I20260926 11:42:49.116603 19816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:42:49.136698 19816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:42:49.139467 19816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:42:49.139635 19816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:42:49.146286 19816 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:43579
I20260926 11:42:49.146297 19883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:43579 every 8 connection(s)
I20260926 11:42:49.147244 19816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:42:49.149395 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 19816
I20260926 11:42:49.149528 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:42:49.149839 6528 external_mini_cluster.cc:1428] Setting key edaea5d852242c6d75e896199f46ba6f
I20260926 11:42:49.150156 19884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:42:49.154071 19884 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43579 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.96.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.61@KRBTEST.COM" created.
Sep 26 11:42:49 dist-test-slave-xxxx krb5kdc[19803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422969, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:42:49.168617 19887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.156270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38936 (local address 127.6.96.62:43579)
0926 11:42:49.156684 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:49.156701 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:49.157039 (+ 338us) server_negotiation.cc:407] Connection header received
0926 11:42:49.158106 (+ 1067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:49.158111 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:49.158309 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:49.158460 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:49.159294 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.160275 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.161250 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.161606 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.161752 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:49.164137 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:49.164144 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:49.164158 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:49.164185 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:49.165840 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.166268 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.166274 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.166276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.166417 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.166715 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.166718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.166720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.167239 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:49.167367 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:49.167696 (+ 329us) server_negotiation.cc:299] Negotiation successful
0926 11:42:49.167907 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":290,"thread_start_us":133,"threads_started":1}
I20260926 11:42:49.168617 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.156261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:43579 (local address 127.0.0.1:38936)
0926 11:42:49.156872 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.156925 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:49.157766 (+ 841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:49.158503 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:49.158508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:49.158870 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:42:49.159146 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:49.159160 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.160396 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.160399 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:49.161132 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:49.161141 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.161745 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.161749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:49.161854 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:49.162398 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:42:49.162417 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:42:49.163994 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:42:49.165943 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:49.165951 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:49.165958 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:49.166167 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:49.166519 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:49.166521 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:49.166523 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:49.166628 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:49.167336 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:42:49.167341 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:42:49.167463 (+ 122us) client_negotiation.cc:769] Sending connection context
0926 11:42:49.167595 (+ 132us) client_negotiation.cc:240] Negotiation successful
0926 11:42:49.167837 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"spinlock_wait_cycles":70912,"thread_start_us":157,"threads_started":1}
I20260926 11:42:49.170959 19884 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } has no permanent_uuid. Determining permanent_uuid...
I20260926 11:42:49.171623 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.171390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:35373 (local address 127.0.0.1:41482)
0926 11:42:49.171501 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.171566 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.61:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 11:42:49.171820 19832 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.96.61:35373: connect: Connection refused (error 111)
I20260926 11:42:49.172629 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.172415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:35373 (local address 127.0.0.1:41496)
0926 11:42:49.172537 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.172580 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.61:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260926 11:42:49.172739 19884 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.61:35373: Network error: Client connection negotiation failed: client connection to 127.6.96.61:35373: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.61.keytab.
Entry for principal kudu/127.6.96.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.61.keytab.
I20260926 11:42:49.177265 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.61:35373
--webserver_interface=127.6.96.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.61.keytab
--principal=kudu/127.6.96.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:36301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:43579,127.6.96.61:35373,127.6.96.60:43653
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 11:42:49.221937 19884 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } attempt: 1
I20260926 11:42:49.222756 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.222477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:35373 (local address 127.0.0.1:41502)
0926 11:42:49.222616 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.222678 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.61:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260926 11:42:49.223420 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.223230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:35373 (local address 127.0.0.1:41514)
0926 11:42:49.223316 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.223357 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.61:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260926 11:42:49.223537 19884 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.61:35373: Network error: Client connection negotiation failed: client connection to 127.6.96.61:35373: connect: Connection refused (error 111)
W20260926 11:42:49.283689 19891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:42:49.284067 19891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:42:49.284132 19891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:42:49.288621 19891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:42:49.288681 19891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:42:49.288699 19891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:42:49.288714 19891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:42:49.288728 19891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:42:49.291700 19884 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } attempt: 2
I20260926 11:42:49.292559 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.292250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:35373 (local address 127.0.0.1:41516)
0926 11:42:49.292418 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.292482 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.61:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260926 11:42:49.293236 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.293015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:35373 (local address 127.0.0.1:41526)
0926 11:42:49.293126 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.293171 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.61:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260926 11:42:49.293342 19884 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.61:35373: Network error: Client connection negotiation failed: client connection to 127.6.96.61:35373: connect: Connection refused (error 111)
I20260926 11:42:49.294368 19891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:36301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:43579,127.6.96.61:35373,127.6.96.60:43653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.61.keytab
--principal=kudu/127.6.96.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.61:35373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.96.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:42:49.295441 19891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:42:49.296446 19891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:42:49.304766 19899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:49.304766 19896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:49.304898 19891 server_base.cc:1034] running on GCE node
W20260926 11:42:49.305044 19897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:49.305563 19891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:42:49.306614 19891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:42:49.307803 19891 hybrid_clock.cc:648] HybridClock initialized: now 1790422969307769 us; error 47 us; skew 500 ppm
Sep 26 11:42:49 dist-test-slave-xxxx krb5kdc[19803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422969, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:49.310688 19891 init.cc:377] Logged in from keytab as kudu/127.6.96.61@KRBTEST.COM (short username kudu)
I20260926 11:42:49.311599 19891 webserver.cc:460] Webserver started at http://127.6.96.61:41533/ using document root <none> and password file <none>
I20260926 11:42:49.312145 19891 fs_manager.cc:359] Metadata directory not provided
I20260926 11:42:49.312216 19891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:42:49.312454 19891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:42:49.314451 19891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-1/data/instance:
uuid: "7ef21d5763834e1ab5c0a2bc52461c7b"
format_stamp: "Formatted at 2026-09-26 11:42:49 on dist-test-slave-xxxx"
server_key: "53941a05c4aca50a8f020090f02eff33"
server_key_iv: "eb36ffeecbde0f690444c99484b49286"
server_key_version: "encryptionkey@0"
I20260926 11:42:49.314998 19891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-1/wal/instance:
uuid: "7ef21d5763834e1ab5c0a2bc52461c7b"
format_stamp: "Formatted at 2026-09-26 11:42:49 on dist-test-slave-xxxx"
server_key: "53941a05c4aca50a8f020090f02eff33"
server_key_iv: "eb36ffeecbde0f690444c99484b49286"
server_key_version: "encryptionkey@0"
I20260926 11:42:49.318564 19891 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 11:42:49.320941 19906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:49.322145 19891 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:42:49.322253 19891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-1/wal
uuid: "7ef21d5763834e1ab5c0a2bc52461c7b"
format_stamp: "Formatted at 2026-09-26 11:42:49 on dist-test-slave-xxxx"
server_key: "53941a05c4aca50a8f020090f02eff33"
server_key_iv: "eb36ffeecbde0f690444c99484b49286"
server_key_version: "encryptionkey@0"
I20260926 11:42:49.322374 19891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:42:49.335757 19891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:42:49.338371 19891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:42:49.338536 19891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:42:49.345022 19891 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.61:35373
I20260926 11:42:49.345038 19958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.61:35373 every 8 connection(s)
I20260926 11:42:49.345894 19891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-1/data/info.pb
I20260926 11:42:49.348568 19959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:42:49.352865 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 19891
I20260926 11:42:49.352998 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-1/wal/instance
I20260926 11:42:49.353271 6528 external_mini_cluster.cc:1428] Setting key 79be302fee868f20a5282abada04d519
I20260926 11:42:49.353461 19959 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43579 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.96.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.60@KRBTEST.COM" created.
Sep 26 11:42:49 dist-test-slave-xxxx krb5kdc[19803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422969, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.61@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:42:49.367913 19887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.355842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38952 (local address 127.6.96.62:43579)
0926 11:42:49.356020 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:49.356024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:49.356943 (+ 919us) server_negotiation.cc:407] Connection header received
0926 11:42:49.358156 (+ 1213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:49.358160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:49.358224 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:49.358323 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:49.359796 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.360330 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.361492 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.361735 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.361825 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:49.364503 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:49.364508 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:49.364510 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:49.364540 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:49.365835 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.366378 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.366382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.366384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.366469 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.366787 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.366790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.366792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.367010 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:49.367100 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:49.367609 (+ 509us) server_negotiation.cc:299] Negotiation successful
0926 11:42:49.367750 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 11:42:49.368647 19962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.356246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:43579 (local address 127.0.0.1:38952)
0926 11:42:49.356783 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.356835 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:49.357902 (+ 1067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:49.358610 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:49.358620 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:49.359114 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:42:49.359647 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:49.359661 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.360457 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.360461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:49.361373 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:49.361383 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.361882 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.361886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:49.362041 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:49.362601 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:42:49.362617 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:42:49.364325 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:42:49.365951 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:49.365956 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:49.365968 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:49.366227 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:49.366554 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:49.366557 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:49.366558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:49.366688 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:49.367108 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:42:49.367114 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:42:49.367363 (+ 249us) client_negotiation.cc:769] Sending connection context
0926 11:42:49.367528 (+ 165us) client_negotiation.cc:240] Negotiation successful
0926 11:42:49.367844 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":120,"threads_started":1}
I20260926 11:42:49.370297 19959 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 11:42:49 dist-test-slave-xxxx krb5kdc[19803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422969, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.61@KRBTEST.COM for kudu/127.6.96.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.60.keytab.
Entry for principal kudu/127.6.96.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.60.keytab.
I20260926 11:42:49.378229 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.60:43653
--webserver_interface=127.6.96.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.60.keytab
--principal=kudu/127.6.96.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:36301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:43579,127.6.96.61:35373,127.6.96.60:43653
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 11:42:49.378746 19962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.370804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:35373 (local address 127.0.0.1:41536)
0926 11:42:49.370950 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.370962 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:49.371025 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:49.371599 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:49.371601 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:49.371749 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:42:49.371823 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:49.371828 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.372882 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.372885 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:49.373393 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:49.373400 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.373956 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.373959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:49.374072 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:49.374477 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:42:49.374488 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:42:49.375624 (+ 1136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:42:49.377227 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:49.377230 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:49.377232 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:49.377423 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:49.377663 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:49.377666 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:49.377668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:49.377744 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:49.378265 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:42:49.378268 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:42:49.378364 (+ 96us) client_negotiation.cc:769] Sending connection context
0926 11:42:49.378472 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 11:42:49.378588 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":1}
I20260926 11:42:49.379323 19964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.370966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41536 (local address 127.6.96.61:35373)
0926 11:42:49.371264 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:49.371272 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:49.371290 (+ 18us) server_negotiation.cc:407] Connection header received
0926 11:42:49.371321 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:49.371325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:49.371495 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:49.371610 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:49.371920 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.372788 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.373489 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.373849 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.373945 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:49.375736 (+ 1791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:49.375745 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:49.375758 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:49.375777 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:49.377132 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.377514 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.377519 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.377520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.377587 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.377867 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.377870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.377871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.378157 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:49.378321 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:49.379025 (+ 704us) server_negotiation.cc:299] Negotiation successful
0926 11:42:49.379154 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":223,"thread_start_us":138,"threads_started":1}
I20260926 11:42:49.380962 19959 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } has no permanent_uuid. Determining permanent_uuid...
I20260926 11:42:49.381630 19962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.381407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:43653 (local address 127.0.0.1:56670)
0926 11:42:49.381512 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.381577 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:43653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 11:42:49.381888 19909 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.96.60:43653: connect: Connection refused (error 111)
I20260926 11:42:49.382706 19962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.382529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:43653 (local address 127.0.0.1:56680)
0926 11:42:49.382622 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.382662 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:43653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260926 11:42:49.382874 19959 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.60:43653: Network error: Client connection negotiation failed: client connection to 127.6.96.60:43653: connect: Connection refused (error 111)
I20260926 11:42:49.384503 19884 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } attempt: 3
Sep 26 11:42:49 dist-test-slave-xxxx krb5kdc[19803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422969, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for kudu/127.6.96.61@KRBTEST.COM
I20260926 11:42:49.393446 19964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.384974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41548 (local address 127.6.96.61:35373)
0926 11:42:49.385067 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:49.385070 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:49.385099 (+ 29us) server_negotiation.cc:407] Connection header received
0926 11:42:49.385235 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:49.385237 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:49.385272 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:49.385341 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:49.386274 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.387160 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.388264 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.388507 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.388583 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:49.390601 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:49.390604 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:49.390606 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:49.390627 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:49.391928 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.392381 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.392384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.392385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.392463 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.392727 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.392730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.392731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.392913 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:49.392995 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:49.393238 (+ 243us) server_negotiation.cc:299] Negotiation successful
0926 11:42:49.393336 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 11:42:49.393481 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.384900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:35373 (local address 127.0.0.1:41548)
0926 11:42:49.385034 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.385051 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:49.385156 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:49.385379 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:49.385382 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:49.385571 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:42:49.386140 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:49.386151 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.387282 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.387286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:49.388146 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:49.388158 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.388602 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.388605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:49.388708 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:49.389113 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:42:49.389127 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:42:49.390481 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:42:49.392014 (+ 1533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:49.392017 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:49.392020 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:49.392266 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:49.392540 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:49.392543 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:49.392545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:49.392626 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:49.392994 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:42:49.392997 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:42:49.393090 (+ 93us) client_negotiation.cc:769] Sending connection context
0926 11:42:49.393184 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 11:42:49.393322 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260926 11:42:49.394259 19884 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } has no permanent_uuid. Determining permanent_uuid...
I20260926 11:42:49.394937 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.394665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:43653 (local address 127.0.0.1:56686)
0926 11:42:49.394782 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.394868 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:43653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260926 11:42:49.395628 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.395388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:43653 (local address 127.0.0.1:56688)
0926 11:42:49.395529 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.395575 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:43653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260926 11:42:49.395780 19884 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.60:43653: Network error: Client connection negotiation failed: client connection to 127.6.96.60:43653: connect: Connection refused (error 111)
I20260926 11:42:49.426930 19884 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } attempt: 1
I20260926 11:42:49.427806 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.427477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:43653 (local address 127.0.0.1:56696)
0926 11:42:49.427637 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.427711 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:43653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260926 11:42:49.428498 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.428289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:43653 (local address 127.0.0.1:56710)
0926 11:42:49.428388 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.428439 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:43653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260926 11:42:49.428619 19884 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.60:43653: Network error: Client connection negotiation failed: client connection to 127.6.96.60:43653: connect: Connection refused (error 111)
I20260926 11:42:49.432120 19959 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } attempt: 1
I20260926 11:42:49.432811 19962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.432556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:43653 (local address 127.0.0.1:56726)
0926 11:42:49.432681 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.432734 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:43653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260926 11:42:49.433451 19962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.433245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:43653 (local address 127.0.0.1:56734)
0926 11:42:49.433347 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.433396 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:43653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260926 11:42:49.433553 19959 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.60:43653: Network error: Client connection negotiation failed: client connection to 127.6.96.60:43653: connect: Connection refused (error 111)
W20260926 11:42:49.489084 19965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:42:49.489362 19965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:42:49.489418 19965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:42:49.493570 19965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:42:49.493639 19965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:42:49.493671 19965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:42:49.493695 19965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:42:49.493716 19965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:42:49.499011 19965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:36301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:43579,127.6.96.61:35373,127.6.96.60:43653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.60.keytab
--principal=kudu/127.6.96.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.60:43653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.96.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:42:49.500015 19965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:42:49.500973 19965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 11:42:49.501704 19959 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } attempt: 2
I20260926 11:42:49.502487 19962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.502210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:43653 (local address 127.0.0.1:56738)
0926 11:42:49.502351 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.502412 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:43653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260926 11:42:49.503161 19962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.502963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:43653 (local address 127.0.0.1:56752)
0926 11:42:49.503061 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.503105 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:43653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260926 11:42:49.503270 19959 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.60:43653: Network error: Client connection negotiation failed: client connection to 127.6.96.60:43653: connect: Connection refused (error 111)
I20260926 11:42:49.503768 19884 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } attempt: 2
I20260926 11:42:49.504443 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.504166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:43653 (local address 127.0.0.1:56766)
0926 11:42:49.504310 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.504368 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:43653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 11:42:49.505139 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.504904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:43653 (local address 127.0.0.1:56772)
0926 11:42:49.505006 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.505079 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:43653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260926 11:42:49.505280 19884 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.60:43653: Network error: Client connection negotiation failed: client connection to 127.6.96.60:43653: connect: Connection refused (error 111)
W20260926 11:42:49.509701 19971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:49.509701 19974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:49.510339 19972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:49.510401 19965 server_base.cc:1034] running on GCE node
I20260926 11:42:49.511008 19965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:42:49.512068 19965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:42:49.513247 19965 hybrid_clock.cc:648] HybridClock initialized: now 1790422969513221 us; error 37 us; skew 500 ppm
Sep 26 11:42:49 dist-test-slave-xxxx krb5kdc[19803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422969, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:49.515969 19965 init.cc:377] Logged in from keytab as kudu/127.6.96.60@KRBTEST.COM (short username kudu)
I20260926 11:42:49.516898 19965 webserver.cc:460] Webserver started at http://127.6.96.60:37429/ using document root <none> and password file <none>
I20260926 11:42:49.517441 19965 fs_manager.cc:359] Metadata directory not provided
I20260926 11:42:49.517505 19965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:42:49.517690 19965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:42:49.519821 19965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-2/data/instance:
uuid: "b927940eeea04f838ad78482d1438a7c"
format_stamp: "Formatted at 2026-09-26 11:42:49 on dist-test-slave-xxxx"
server_key: "ec93223c49c9b7f508089a5f2145dbec"
server_key_iv: "d5ea647dcea70612a5aeeed0f216862e"
server_key_version: "encryptionkey@0"
I20260926 11:42:49.520358 19965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-2/wal/instance:
uuid: "b927940eeea04f838ad78482d1438a7c"
format_stamp: "Formatted at 2026-09-26 11:42:49 on dist-test-slave-xxxx"
server_key: "ec93223c49c9b7f508089a5f2145dbec"
server_key_iv: "d5ea647dcea70612a5aeeed0f216862e"
server_key_version: "encryptionkey@0"
I20260926 11:42:49.523921 19965 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260926 11:42:49.526254 19981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:49.527552 19965 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:42:49.527668 19965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-2/wal
uuid: "b927940eeea04f838ad78482d1438a7c"
format_stamp: "Formatted at 2026-09-26 11:42:49 on dist-test-slave-xxxx"
server_key: "ec93223c49c9b7f508089a5f2145dbec"
server_key_iv: "d5ea647dcea70612a5aeeed0f216862e"
server_key_version: "encryptionkey@0"
I20260926 11:42:49.527783 19965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:42:49.559101 19965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:42:49.561965 19965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:42:49.562148 19965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:42:49.568876 19965 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.60:43653
I20260926 11:42:49.568907 20033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.60:43653 every 8 connection(s)
I20260926 11:42:49.569854 19965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-2/data/info.pb
I20260926 11:42:49.572629 20034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:42:49.574697 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 19965
I20260926 11:42:49.574867 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790422817500949-6528-0/minicluster-data/master-2/wal/instance
I20260926 11:42:49.575129 6528 external_mini_cluster.cc:1428] Setting key c6b9081663e39ddf2222b0750b6ff1c6
I20260926 11:42:49.576625 20034 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43579 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 11:42:49 dist-test-slave-xxxx krb5kdc[19803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422969, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.60@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
Sep 26 11:42:49 dist-test-slave-xxxx krb5kdc[19803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.60@KRBTEST.COM
I20260926 11:42:49.588433 19887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.578281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38958 (local address 127.6.96.62:43579)
0926 11:42:49.578411 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:49.578416 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:49.578791 (+ 375us) server_negotiation.cc:407] Connection header received
0926 11:42:49.578926 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:49.578929 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:49.578983 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:49.579042 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:49.579918 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.580491 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.581712 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.581972 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.582080 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:49.584388 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:49.584392 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:49.584394 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:49.584419 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:49.586021 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.586774 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.586778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.586780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.586893 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.587253 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.587256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.587258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.587534 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:49.587628 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:49.588131 (+ 503us) server_negotiation.cc:299] Negotiation successful
0926 11:42:49.588254 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 11:42:49.589272 20038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.578233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:43579 (local address 127.0.0.1:38958)
0926 11:42:49.578642 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.578689 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:49.578809 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:49.579101 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:49.579106 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:49.579566 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:42:49.579759 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:49.579771 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.580615 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.580617 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:49.581580 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:49.581590 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.582115 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.582117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:49.582190 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:49.582690 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:42:49.582705 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:42:49.584225 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:42:49.586171 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:49.586178 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:49.586194 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:49.586672 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:49.586976 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:49.586979 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:49.586981 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:49.587169 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:49.587630 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:42:49.587635 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:42:49.587896 (+ 261us) client_negotiation.cc:769] Sending connection context
0926 11:42:49.588064 (+ 168us) client_negotiation.cc:240] Negotiation successful
0926 11:42:49.588395 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":36,"spinlock_wait_cycles":3456,"thread_start_us":76,"threads_started":1}
I20260926 11:42:49.589272 20037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.576437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56776 (local address 127.6.96.60:43653)
0926 11:42:49.576857 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:49.576875 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:49.576911 (+ 36us) server_negotiation.cc:407] Connection header received
0926 11:42:49.577553 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:49.577571 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:49.577908 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:49.578197 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:49.578997 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.580218 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.581144 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.581800 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.581916 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:49.584798 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:49.584807 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:49.584822 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:49.584848 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:49.586681 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.587121 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.587126 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.587127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.587195 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.587452 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.587456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.587458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.587879 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:49.588020 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:49.588280 (+ 260us) server_negotiation.cc:299] Negotiation successful
0926 11:42:49.588444 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":278,"thread_start_us":127,"threads_started":1}
I20260926 11:42:49.591282 20034 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } has no permanent_uuid. Determining permanent_uuid...
I20260926 11:42:49.594411 19959 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } attempt: 3
Sep 26 11:42:49 dist-test-slave-xxxx krb5kdc[19803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422969, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.60@KRBTEST.COM for kudu/127.6.96.61@KRBTEST.COM
Sep 26 11:42:49 dist-test-slave-xxxx krb5kdc[19803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
Sep 26 11:42:49 dist-test-slave-xxxx krb5kdc[19803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422969, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.61@KRBTEST.COM for kudu/127.6.96.60@KRBTEST.COM
I20260926 11:42:49.600154 19964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.591871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41550 (local address 127.6.96.61:35373)
0926 11:42:49.592090 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:49.592095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:49.592106 (+ 11us) server_negotiation.cc:407] Connection header received
0926 11:42:49.592142 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:49.592144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:49.592181 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:49.592319 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:49.592712 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.593457 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.594544 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.594897 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.594994 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:49.596856 (+ 1862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:49.596860 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:49.596863 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:49.596889 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:49.598588 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.599046 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.599050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.599052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.599134 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.599378 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.599382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.599383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.599555 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:49.599638 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:49.599927 (+ 289us) server_negotiation.cc:299] Negotiation successful
0926 11:42:49.600027 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260926 11:42:49.600139 20038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.591794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:35373 (local address 127.0.0.1:41550)
0926 11:42:49.591908 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.591922 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:49.591993 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:49.592327 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:49.592329 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:49.592488 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:42:49.592590 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:49.592597 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.593576 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.593578 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:49.594430 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:49.594440 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.594987 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.594991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:49.595085 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:49.595532 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:42:49.595545 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:42:49.596748 (+ 1203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:42:49.598704 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:49.598707 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:49.598710 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:49.598957 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:49.599213 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:49.599216 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:49.599218 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:49.599302 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:49.599677 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:42:49.599679 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:42:49.599751 (+ 72us) client_negotiation.cc:769] Sending connection context
0926 11:42:49.599848 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 11:42:49.599976 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260926 11:42:49.601195 20034 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } has no permanent_uuid. Determining permanent_uuid...
I20260926 11:42:49.601366 19887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.591979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38966 (local address 127.6.96.62:43579)
0926 11:42:49.592086 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:49.592090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:49.592104 (+ 14us) server_negotiation.cc:407] Connection header received
0926 11:42:49.592139 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:49.592141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:49.592181 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:49.592254 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:49.592939 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.593613 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.594637 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.594942 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.595032 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:49.597565 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:49.597569 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:49.597572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:49.597596 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:49.599431 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.599876 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.599880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.599882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.599966 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.600221 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.600225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.600227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.600420 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:49.600640 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:49.600963 (+ 323us) server_negotiation.cc:299] Negotiation successful
0926 11:42:49.601190 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 11:42:49.603840 19962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.594787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:43653 (local address 127.0.0.1:56792)
0926 11:42:49.594975 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.594988 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:49.595044 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:49.595396 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:49.595400 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:49.595573 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:42:49.595670 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:49.595678 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.596802 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.596805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:49.597527 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:49.597540 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.598044 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.598047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:49.598130 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:49.598484 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:42:49.598498 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:42:49.599834 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:42:49.602318 (+ 2484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:49.602321 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:49.602323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:49.602583 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:49.602894 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:49.602898 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:49.602899 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:49.602984 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:49.603370 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:42:49.603373 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:42:49.603460 (+ 87us) client_negotiation.cc:769] Sending connection context
0926 11:42:49.603556 (+ 96us) client_negotiation.cc:240] Negotiation successful
0926 11:42:49.603686 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260926 11:42:49.603855 20037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.594868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56792 (local address 127.6.96.60:43653)
0926 11:42:49.595134 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:49.595138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:49.595149 (+ 11us) server_negotiation.cc:407] Connection header received
0926 11:42:49.595178 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:49.595181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:49.595215 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:49.595321 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:49.595791 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.596624 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.597632 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.597940 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.598026 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:49.599960 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:49.599964 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:49.599966 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:49.599986 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:49.602229 (+ 2243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.602681 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.602683 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.602685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.602760 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.603087 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.603089 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.603091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.603273 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:49.603365 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:49.603611 (+ 246us) server_negotiation.cc:299] Negotiation successful
0926 11:42:49.603707 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201}
I20260926 11:42:49.604425 19884 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } attempt: 3
Sep 26 11:42:49 dist-test-slave-xxxx krb5kdc[19803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422969, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.60@KRBTEST.COM for kudu/127.6.96.60@KRBTEST.COM
I20260926 11:42:49.607730 19959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ef21d5763834e1ab5c0a2bc52461c7b: Bootstrap starting.
Sep 26 11:42:49 dist-test-slave-xxxx krb5kdc[19803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.61@KRBTEST.COM
I20260926 11:42:49.610702 19959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ef21d5763834e1ab5c0a2bc52461c7b: Neither blocks nor log segments found. Creating new log.
I20260926 11:42:49.611346 20038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.601687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:43653 (local address 127.0.0.1:56808)
0926 11:42:49.602376 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.602388 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:49.602454 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:49.602734 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:49.602737 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:49.602901 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:42:49.602985 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:49.602992 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.603936 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.603939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:49.605441 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:49.605449 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.605909 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.605913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:49.606038 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:49.606398 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:42:49.606411 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:42:49.608250 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:42:49.609905 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:49.609908 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:49.609910 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:49.610075 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:49.610295 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:49.610298 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:49.610300 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:49.610376 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:49.610861 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:42:49.610864 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:42:49.610949 (+ 85us) client_negotiation.cc:769] Sending connection context
0926 11:42:49.611031 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 11:42:49.611165 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"mutex_wait_us":21}
I20260926 11:42:49.611358 20039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.601779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56808 (local address 127.6.96.60:43653)
0926 11:42:49.602062 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:49.602068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:49.602519 (+ 451us) server_negotiation.cc:407] Connection header received
0926 11:42:49.602544 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:49.602546 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:49.602580 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:49.602646 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:49.603081 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.603836 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.605529 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.605807 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.605903 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:49.608413 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:49.608418 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:49.608421 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:49.608443 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:49.609801 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.610149 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.610153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.610155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.610224 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.610447 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.610450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.610451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.610706 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:49.610786 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:49.611084 (+ 298us) server_negotiation.cc:299] Negotiation successful
0926 11:42:49.611217 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":229,"thread_start_us":96,"threads_started":1}
I20260926 11:42:49.611666 19959 log.cc:824] T 00000000000000000000000000000000 P 7ef21d5763834e1ab5c0a2bc52461c7b: Log is configured to *not* fsync() on all Append() calls
I20260926 11:42:49.611881 19964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.602535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41562 (local address 127.6.96.61:35373)
0926 11:42:49.602629 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:49.602633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:49.602689 (+ 56us) server_negotiation.cc:407] Connection header received
0926 11:42:49.602813 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:49.602816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:49.602886 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:49.602948 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:49.603536 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.604232 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.605379 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.605663 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.605765 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:49.608435 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:49.608438 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:49.608441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:49.608464 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:49.610030 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.610504 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.610507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.610509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.610580 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.610914 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.610918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.610919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.611136 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:49.611298 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:49.611649 (+ 351us) server_negotiation.cc:299] Negotiation successful
0926 11:42:49.611756 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 11:42:49.614202 19959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ef21d5763834e1ab5c0a2bc52461c7b: No bootstrap required, opened a new log
I20260926 11:42:49.614651 20034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b927940eeea04f838ad78482d1438a7c: Bootstrap starting.
I20260926 11:42:49.615195 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 11:42:49.617352 20034 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b927940eeea04f838ad78482d1438a7c: Neither blocks nor log segments found. Creating new log.
I20260926 11:42:49.618239 20034 log.cc:824] T 00000000000000000000000000000000 P b927940eeea04f838ad78482d1438a7c: Log is configured to *not* fsync() on all Append() calls
I20260926 11:42:49.618220 19959 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ef21d5763834e1ab5c0a2bc52461c7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5a4c708307474a82f00458ac47dbb4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43579 } } peers { permanent_uuid: "7ef21d5763834e1ab5c0a2bc52461c7b" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } } peers { permanent_uuid: "b927940eeea04f838ad78482d1438a7c" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } }
I20260926 11:42:49.620589 20034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b927940eeea04f838ad78482d1438a7c: No bootstrap required, opened a new log
I20260926 11:42:49.621835 19959 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ef21d5763834e1ab5c0a2bc52461c7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Sep 26 11:42:49 dist-test-slave-xxxx krb5kdc[19803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422969, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for kudu/127.6.96.60@KRBTEST.COM
I20260926 11:42:49.623981 19959 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ef21d5763834e1ab5c0a2bc52461c7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ef21d5763834e1ab5c0a2bc52461c7b, State: Initialized, Role: FOLLOWER
I20260926 11:42:49.624609 20034 raft_consensus.cc:348] T 00000000000000000000000000000000 P b927940eeea04f838ad78482d1438a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5a4c708307474a82f00458ac47dbb4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43579 } } peers { permanent_uuid: "7ef21d5763834e1ab5c0a2bc52461c7b" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } } peers { permanent_uuid: "b927940eeea04f838ad78482d1438a7c" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } }
I20260926 11:42:49.624863 20034 raft_consensus.cc:374] T 00000000000000000000000000000000 P b927940eeea04f838ad78482d1438a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:42:49.624912 20034 raft_consensus.cc:729] T 00000000000000000000000000000000 P b927940eeea04f838ad78482d1438a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b927940eeea04f838ad78482d1438a7c, State: Initialized, Role: FOLLOWER
I20260926 11:42:49.625452 20034 consensus_queue.cc:260] T 00000000000000000000000000000000 P b927940eeea04f838ad78482d1438a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5a4c708307474a82f00458ac47dbb4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43579 } } peers { permanent_uuid: "7ef21d5763834e1ab5c0a2bc52461c7b" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } } peers { permanent_uuid: "b927940eeea04f838ad78482d1438a7c" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } }
I20260926 11:42:49.626946 20034 sys_catalog.cc:564] T 00000000000000000000000000000000 P b927940eeea04f838ad78482d1438a7c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:42:49.627476 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.604762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:43653 (local address 127.0.0.1:56812)
0926 11:42:49.604913 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:49.604924 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:49.604986 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:49.618577 (+ 13591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:49.618580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:49.618771 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:42:49.618891 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:49.618900 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.619960 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.619964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:49.620714 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:49.620725 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.621372 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.621376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:49.621455 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:49.621973 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:42:49.621988 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:42:49.623569 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:42:49.625894 (+ 2325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:49.625897 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:49.625899 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:49.626125 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:49.626428 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:49.626431 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:49.626433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:49.626537 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:49.627000 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:42:49.627003 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:42:49.627096 (+ 93us) client_negotiation.cc:769] Sending connection context
0926 11:42:49.627178 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 11:42:49.627306 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260926 11:42:49.628074 19964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.617040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41574 (local address 127.6.96.61:35373)
0926 11:42:49.617143 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:49.617148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:49.618051 (+ 903us) server_negotiation.cc:407] Connection header received
0926 11:42:49.618243 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:49.618246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:49.618290 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:49.618352 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:49.619237 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.619995 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.621214 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.621527 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.621595 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:49.623553 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:49.623557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:49.623560 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:49.623584 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:49.625933 (+ 2349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.626393 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.626396 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.626398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.626481 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.626751 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.626754 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.626756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.626993 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:49.627108 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:49.627442 (+ 334us) server_negotiation.cc:299] Negotiation successful
0926 11:42:49.627548 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 11:42:49.628458 19959 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ef21d5763834e1ab5c0a2bc52461c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5a4c708307474a82f00458ac47dbb4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43579 } } peers { permanent_uuid: "7ef21d5763834e1ab5c0a2bc52461c7b" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } } peers { permanent_uuid: "b927940eeea04f838ad78482d1438a7c" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } }
I20260926 11:42:49.630015 19959 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ef21d5763834e1ab5c0a2bc52461c7b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:42:49.632066 20037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.604809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56812 (local address 127.6.96.60:43653)
0926 11:42:49.612543 (+ 7734us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:49.612547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:49.612560 (+ 13us) server_negotiation.cc:407] Connection header received
0926 11:42:49.612591 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:49.612593 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:49.612634 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:49.619014 (+ 6380us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:49.619099 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.619826 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.620937 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.621235 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.621681 (+ 446us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:49.623709 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:49.623712 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:49.623715 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:49.623738 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:49.625781 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.626235 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.626238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.626240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.626327 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.626666 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.626669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.626670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.626892 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:49.627973 (+ 1081us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:49.628050 (+ 77us) server_negotiation.cc:299] Negotiation successful
0926 11:42:49.631923 (+ 3873us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7357}
I20260926 11:42:49.632758 19887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.617416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38982 (local address 127.6.96.62:43579)
0926 11:42:49.617524 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:49.617528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:49.618063 (+ 535us) server_negotiation.cc:407] Connection header received
0926 11:42:49.618478 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:49.618481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:49.618528 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:49.618577 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:49.620273 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.621030 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.624007 (+ 2977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.624335 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.624409 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:49.627331 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:49.627335 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:49.627338 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:49.627363 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:49.629158 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.631463 (+ 2305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.631467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.631469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.631547 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.631856 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.631860 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.631862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.632056 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:49.632157 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:49.632513 (+ 356us) server_negotiation.cc:299] Negotiation successful
0926 11:42:49.632622 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 11:42:49.630627 19965 master_runner.cc:186] Error getting master registration for 127.6.96.62:43579: OK, instance_id { permanent_uuid: "ec5a4c708307474a82f00458ac47dbb4" instance_seqno: 1790422969144403 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 11:42:49.633916 19965 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 11:42:49.634090 20061 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b927940eeea04f838ad78482d1438a7c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 11:42:49.634266 20061 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 11:42:49.635028 19884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4: Bootstrap starting.
I20260926 11:42:49.635218 20060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ef21d5763834e1ab5c0a2bc52461c7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5a4c708307474a82f00458ac47dbb4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43579 } } peers { permanent_uuid: "7ef21d5763834e1ab5c0a2bc52461c7b" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } } peers { permanent_uuid: "b927940eeea04f838ad78482d1438a7c" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } } }
I20260926 11:42:49.635305 20039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:49.617706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56824 (local address 127.6.96.60:43653)
0926 11:42:49.617795 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:49.617799 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:49.618237 (+ 438us) server_negotiation.cc:407] Connection header received
0926 11:42:49.618705 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:49.618709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:49.618751 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:49.618798 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:49.622220 (+ 3422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.622998 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.624885 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:49.625237 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:49.625308 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:49.627666 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:49.627670 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:49.627672 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:49.627696 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:49.630680 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.634031 (+ 3351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.634034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.634036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.634121 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:49.634446 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:49.634450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:49.634452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:49.634654 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:49.634805 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:49.635065 (+ 260us) server_negotiation.cc:299] Negotiation successful
0926 11:42:49.635177 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 11:42:49.635349 20060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ef21d5763834e1ab5c0a2bc52461c7b [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:42:49.635890 20049 sys_catalog.cc:455] T 00000000000000000000000000000000 P b927940eeea04f838ad78482d1438a7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5a4c708307474a82f00458ac47dbb4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43579 } } peers { permanent_uuid: "7ef21d5763834e1ab5c0a2bc52461c7b" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } } peers { permanent_uuid: "b927940eeea04f838ad78482d1438a7c" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } } }
I20260926 11:42:49.635998 20049 sys_catalog.cc:458] T 00000000000000000000000000000000 P b927940eeea04f838ad78482d1438a7c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:42:49.637339 19884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4: Neither blocks nor log segments found. Creating new log.
I20260926 11:42:49.637966 19884 log.cc:824] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4: Log is configured to *not* fsync() on all Append() calls
W20260926 11:42:49.640699 20072 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7ef21d5763834e1ab5c0a2bc52461c7b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 11:42:49.640805 20072 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 11:42:49.640841 19884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4: No bootstrap required, opened a new log
I20260926 11:42:49.639780 19891 master_runner.cc:186] Error getting master registration for 127.6.96.62:43579: OK, instance_id { permanent_uuid: "ec5a4c708307474a82f00458ac47dbb4" instance_seqno: 1790422969144403 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 11:42:49.643615 19891 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 11:42:49.643867 19884 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5a4c708307474a82f00458ac47dbb4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43579 } } peers { permanent_uuid: "7ef21d5763834e1ab5c0a2bc52461c7b" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } } peers { permanent_uuid: "b927940eeea04f838ad78482d1438a7c" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } }
I20260926 11:42:49.644590 19884 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:42:49.644683 19884 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec5a4c708307474a82f00458ac47dbb4, State: Initialized, Role: FOLLOWER
I20260926 11:42:49.645107 19884 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5a4c708307474a82f00458ac47dbb4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43579 } } peers { permanent_uuid: "7ef21d5763834e1ab5c0a2bc52461c7b" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } } peers { permanent_uuid: "b927940eeea04f838ad78482d1438a7c" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } }
I20260926 11:42:49.645512 20076 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5a4c708307474a82f00458ac47dbb4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43579 } } peers { permanent_uuid: "7ef21d5763834e1ab5c0a2bc52461c7b" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } } peers { permanent_uuid: "b927940eeea04f838ad78482d1438a7c" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } } }
I20260926 11:42:49.645640 20076 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:42:49.646240 19884 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 11:42:49.651791 20087 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 11:42:49.651911 20087 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 11:42:49.655258 19816 master_runner.cc:263] No leader master found from master ec5a4c708307474a82f00458ac47dbb4
I20260926 11:42:49.655324 19816 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 11:42:49.741833 20076 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:42:49.742030 20076 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5a4c708307474a82f00458ac47dbb4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43579 } } peers { permanent_uuid: "7ef21d5763834e1ab5c0a2bc52461c7b" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } } peers { permanent_uuid: "b927940eeea04f838ad78482d1438a7c" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } }
I20260926 11:42:49.743114 20076 leader_election.cc:288] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ef21d5763834e1ab5c0a2bc52461c7b (127.6.96.61:35373), b927940eeea04f838ad78482d1438a7c (127.6.96.60:43653)
I20260926 11:42:49.744028 20008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ec5a4c708307474a82f00458ac47dbb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b927940eeea04f838ad78482d1438a7c" is_pre_election: true
I20260926 11:42:49.744350 20008 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b927940eeea04f838ad78482d1438a7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec5a4c708307474a82f00458ac47dbb4 in term 0.
I20260926 11:42:49.745023 19933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ec5a4c708307474a82f00458ac47dbb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ef21d5763834e1ab5c0a2bc52461c7b" is_pre_election: true
I20260926 11:42:49.745275 19933 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7ef21d5763834e1ab5c0a2bc52461c7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec5a4c708307474a82f00458ac47dbb4 in term 0.
I20260926 11:42:49.745783 19832 leader_election.cc:302] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ef21d5763834e1ab5c0a2bc52461c7b, ec5a4c708307474a82f00458ac47dbb4; no voters:
I20260926 11:42:49.746146 20076 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:42:49.746238 20076 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:42:49.746279 20076 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:42:49.747645 20076 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5a4c708307474a82f00458ac47dbb4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43579 } } peers { permanent_uuid: "7ef21d5763834e1ab5c0a2bc52461c7b" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } } peers { permanent_uuid: "b927940eeea04f838ad78482d1438a7c" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } }
I20260926 11:42:49.748005 20076 leader_election.cc:288] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [CANDIDATE]: Term 1 election: Requested vote from peers 7ef21d5763834e1ab5c0a2bc52461c7b (127.6.96.61:35373), b927940eeea04f838ad78482d1438a7c (127.6.96.60:43653)
I20260926 11:42:49.748898 20008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ec5a4c708307474a82f00458ac47dbb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b927940eeea04f838ad78482d1438a7c"
I20260926 11:42:49.749025 20008 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b927940eeea04f838ad78482d1438a7c [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:42:49.750679 20008 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b927940eeea04f838ad78482d1438a7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec5a4c708307474a82f00458ac47dbb4 in term 1.
I20260926 11:42:49.751101 19933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ec5a4c708307474a82f00458ac47dbb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ef21d5763834e1ab5c0a2bc52461c7b"
I20260926 11:42:49.751219 19933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ef21d5763834e1ab5c0a2bc52461c7b [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:42:49.752431 19933 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7ef21d5763834e1ab5c0a2bc52461c7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec5a4c708307474a82f00458ac47dbb4 in term 1.
I20260926 11:42:49.752887 19834 leader_election.cc:302] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b927940eeea04f838ad78482d1438a7c, ec5a4c708307474a82f00458ac47dbb4; no voters:
I20260926 11:42:49.753334 20076 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:42:49.753610 20076 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [term 1 LEADER]: Becoming Leader. State: Replica: ec5a4c708307474a82f00458ac47dbb4, State: Running, Role: LEADER
I20260926 11:42:49.753997 20076 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5a4c708307474a82f00458ac47dbb4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43579 } } peers { permanent_uuid: "7ef21d5763834e1ab5c0a2bc52461c7b" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } } peers { permanent_uuid: "b927940eeea04f838ad78482d1438a7c" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } }
I20260926 11:42:49.758167 20076 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec5a4c708307474a82f00458ac47dbb4. Latest consensus state: current_term: 1 leader_uuid: "ec5a4c708307474a82f00458ac47dbb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5a4c708307474a82f00458ac47dbb4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43579 } } peers { permanent_uuid: "7ef21d5763834e1ab5c0a2bc52461c7b" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } } peers { permanent_uuid: "b927940eeea04f838ad78482d1438a7c" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } } }
I20260926 11:42:49.759284 20076 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [sys.catalog]: This master's current role is: LEADER
I20260926 11:42:49.759728 20091 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:42:49.760706 20091 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:42:49.768761 20008 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b927940eeea04f838ad78482d1438a7c [term 1 FOLLOWER]: Refusing update from remote peer ec5a4c708307474a82f00458ac47dbb4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:42:49.769701 20089 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b927940eeea04f838ad78482d1438a7c" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:42:49.771056 19933 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7ef21d5763834e1ab5c0a2bc52461c7b [term 1 FOLLOWER]: Refusing update from remote peer ec5a4c708307474a82f00458ac47dbb4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:42:49.771986 20076 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ef21d5763834e1ab5c0a2bc52461c7b" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:42:49.782069 20049 sys_catalog.cc:455] T 00000000000000000000000000000000 P b927940eeea04f838ad78482d1438a7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec5a4c708307474a82f00458ac47dbb4. Latest consensus state: current_term: 1 leader_uuid: "ec5a4c708307474a82f00458ac47dbb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5a4c708307474a82f00458ac47dbb4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43579 } } peers { permanent_uuid: "7ef21d5763834e1ab5c0a2bc52461c7b" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } } peers { permanent_uuid: "b927940eeea04f838ad78482d1438a7c" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } } }
I20260926 11:42:49.782202 20049 sys_catalog.cc:458] T 00000000000000000000000000000000 P b927940eeea04f838ad78482d1438a7c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:42:49.783820 20060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ef21d5763834e1ab5c0a2bc52461c7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec5a4c708307474a82f00458ac47dbb4. Latest consensus state: current_term: 1 leader_uuid: "ec5a4c708307474a82f00458ac47dbb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5a4c708307474a82f00458ac47dbb4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43579 } } peers { permanent_uuid: "7ef21d5763834e1ab5c0a2bc52461c7b" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } } peers { permanent_uuid: "b927940eeea04f838ad78482d1438a7c" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } } }
I20260926 11:42:49.783936 20060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ef21d5763834e1ab5c0a2bc52461c7b [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:42:49.788470 20089 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec5a4c708307474a82f00458ac47dbb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5a4c708307474a82f00458ac47dbb4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43579 } } peers { permanent_uuid: "7ef21d5763834e1ab5c0a2bc52461c7b" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } } peers { permanent_uuid: "b927940eeea04f838ad78482d1438a7c" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } } }
I20260926 11:42:49.788616 20089 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [sys.catalog]: This master's current role is: LEADER
I20260926 11:42:49.788981 20060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ef21d5763834e1ab5c0a2bc52461c7b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec5a4c708307474a82f00458ac47dbb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5a4c708307474a82f00458ac47dbb4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43579 } } peers { permanent_uuid: "7ef21d5763834e1ab5c0a2bc52461c7b" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } } peers { permanent_uuid: "b927940eeea04f838ad78482d1438a7c" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } } }
I20260926 11:42:49.789096 20060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ef21d5763834e1ab5c0a2bc52461c7b [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:42:49.789559 20092 mvcc.cc:205] Tried to move back new op lower bound from 7333572484160188416 to 7333572484124991488. Current Snapshot: MvccSnapshot[applied={T|T < 7333572484160188416}]
I20260926 11:42:49.793329 20094 mvcc.cc:205] Tried to move back new op lower bound from 7333572484160188416 to 7333572484124991488. Current Snapshot: MvccSnapshot[applied={T|T < 7333572484160188416}]
I20260926 11:42:49.794209 20076 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec5a4c708307474a82f00458ac47dbb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5a4c708307474a82f00458ac47dbb4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43579 } } peers { permanent_uuid: "7ef21d5763834e1ab5c0a2bc52461c7b" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } } peers { permanent_uuid: "b927940eeea04f838ad78482d1438a7c" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } } }
I20260926 11:42:49.794330 20076 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4 [sys.catalog]: This master's current role is: LEADER
I20260926 11:42:49.795802 20049 sys_catalog.cc:455] T 00000000000000000000000000000000 P b927940eeea04f838ad78482d1438a7c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec5a4c708307474a82f00458ac47dbb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5a4c708307474a82f00458ac47dbb4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 43579 } } peers { permanent_uuid: "7ef21d5763834e1ab5c0a2bc52461c7b" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 35373 } } peers { permanent_uuid: "b927940eeea04f838ad78482d1438a7c" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 43653 } } }
I20260926 11:42:49.795922 20049 sys_catalog.cc:458] T 00000000000000000000000000000000 P b927940eeea04f838ad78482d1438a7c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:42:49.796759 20093 mvcc.cc:205] Tried to move back new op lower bound from 7333572484160188416 to 7333572484124991488. Current Snapshot: MvccSnapshot[applied={T|T < 7333572484160188416}]
I20260926 11:42:49.800525 20091 catalog_manager.cc:1347] Generated new cluster ID: f3a706920d6545c7a57a425ffe4f1360
I20260926 11:42:49.800586 20091 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:42:49.831756 20091 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:42:49.833843 20091 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:42:49.845875 20091 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec5a4c708307474a82f00458ac47dbb4: Generated new TSK 0
I20260926 11:42:49.846859 20091 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:42:50.635691 20061 catalog_manager.cc:1259] Loaded cluster ID: f3a706920d6545c7a57a425ffe4f1360
I20260926 11:42:50.635777 20061 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b927940eeea04f838ad78482d1438a7c: loading cluster ID for follower catalog manager: success
I20260926 11:42:50.637480 20061 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b927940eeea04f838ad78482d1438a7c: acquiring CA information for follower catalog manager: success
I20260926 11:42:50.638700 20061 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b927940eeea04f838ad78482d1438a7c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 11:42:50.641944 20072 catalog_manager.cc:1259] Loaded cluster ID: f3a706920d6545c7a57a425ffe4f1360
I20260926 11:42:50.642006 20072 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7ef21d5763834e1ab5c0a2bc52461c7b: loading cluster ID for follower catalog manager: success
I20260926 11:42:50.643643 20072 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7ef21d5763834e1ab5c0a2bc52461c7b: acquiring CA information for follower catalog manager: success
I20260926 11:42:50.644744 20072 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7ef21d5763834e1ab5c0a2bc52461c7b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 11:42:50.919150 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 19816
I20260926 11:42:50.924717 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 19891
I20260926 11:42:50.929962 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 19965
2026-09-26T11:42:50Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3034 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:42:50 dist-test-slave-xxxx krb5kdc[20106](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:42:50 dist-test-slave-xxxx krb5kdc[20106](info): set up 2 sockets
Sep 26 11:42:50 dist-test-slave-xxxx krb5kdc[20106](info): commencing operation
krb5kdc: starting...
W20260926 11:42:51.874759 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.925s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:42:51 dist-test-slave-xxxx krb5kdc[20106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:42:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:42:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:42:51.993943 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:40147
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:34443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:40147
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:42:52.103350 20119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:42:52.103624 20119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:42:52.103689 20119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:42:52.107772 20119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:42:52.107838 20119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:42:52.107863 20119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:42:52.107885 20119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:42:52.107913 20119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:42:52.113365 20119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:34443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:40147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:40147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:42:52.114459 20119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:42:52.115535 20119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:42:52.124202 20125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:52.124317 20119 server_base.cc:1034] running on GCE node
W20260926 11:42:52.124171 20124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:52.124181 20127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:52.125094 20119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:42:52.126241 20119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:42:52.127391 20119 hybrid_clock.cc:648] HybridClock initialized: now 1790422972127370 us; error 38 us; skew 500 ppm
Sep 26 11:42:52 dist-test-slave-xxxx krb5kdc[20106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422972, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:52.130180 20119 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:42:52.131135 20119 webserver.cc:460] Webserver started at http://127.6.96.62:36651/ using document root <none> and password file <none>
I20260926 11:42:52.131665 20119 fs_manager.cc:359] Metadata directory not provided
I20260926 11:42:52.131722 20119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:42:52.131891 20119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:42:52.133927 20119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "a8f896f9b55f4c5ab6c94d295b302394"
format_stamp: "Formatted at 2026-09-26 11:42:52 on dist-test-slave-xxxx"
server_key: "d38b6898d85a625ebeaa4c011384d25e"
server_key_iv: "7e0a2f00567e0a7fb3a5e7ee70ca598d"
server_key_version: "encryptionkey@0"
I20260926 11:42:52.134430 20119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "a8f896f9b55f4c5ab6c94d295b302394"
format_stamp: "Formatted at 2026-09-26 11:42:52 on dist-test-slave-xxxx"
server_key: "d38b6898d85a625ebeaa4c011384d25e"
server_key_iv: "7e0a2f00567e0a7fb3a5e7ee70ca598d"
server_key_version: "encryptionkey@0"
I20260926 11:42:52.138170 20119 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 11:42:52.140682 20134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:52.142023 20119 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:42:52.142144 20119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "a8f896f9b55f4c5ab6c94d295b302394"
format_stamp: "Formatted at 2026-09-26 11:42:52 on dist-test-slave-xxxx"
server_key: "d38b6898d85a625ebeaa4c011384d25e"
server_key_iv: "7e0a2f00567e0a7fb3a5e7ee70ca598d"
server_key_version: "encryptionkey@0"
I20260926 11:42:52.142267 20119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:42:52.160697 20119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:42:52.163686 20119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:42:52.163861 20119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:42:52.170612 20119 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:40147
I20260926 11:42:52.170652 20186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:40147 every 8 connection(s)
I20260926 11:42:52.171581 20119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:42:52.174597 20187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:42:52.179934 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 20119
I20260926 11:42:52.180043 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:42:52.180110 20187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8f896f9b55f4c5ab6c94d295b302394: Bootstrap starting.
I20260926 11:42:52.180320 6528 external_mini_cluster.cc:1428] Setting key f9a142b2f27048749480662b39aef874
I20260926 11:42:52.182580 20187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8f896f9b55f4c5ab6c94d295b302394: Neither blocks nor log segments found. Creating new log.
I20260926 11:42:52.183362 20187 log.cc:824] T 00000000000000000000000000000000 P a8f896f9b55f4c5ab6c94d295b302394: Log is configured to *not* fsync() on all Append() calls
I20260926 11:42:52.185154 20187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8f896f9b55f4c5ab6c94d295b302394: No bootstrap required, opened a new log
Sep 26 11:42:52 dist-test-slave-xxxx krb5kdc[20106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:42:52.188752 20187 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8f896f9b55f4c5ab6c94d295b302394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f896f9b55f4c5ab6c94d295b302394" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40147 } }
I20260926 11:42:52.189015 20187 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8f896f9b55f4c5ab6c94d295b302394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:42:52.189078 20187 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8f896f9b55f4c5ab6c94d295b302394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8f896f9b55f4c5ab6c94d295b302394, State: Initialized, Role: FOLLOWER
I20260926 11:42:52.189549 20187 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8f896f9b55f4c5ab6c94d295b302394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f896f9b55f4c5ab6c94d295b302394" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40147 } }
I20260926 11:42:52.189688 20187 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8f896f9b55f4c5ab6c94d295b302394 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:42:52.189756 20187 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8f896f9b55f4c5ab6c94d295b302394 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:42:52.189855 20187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8f896f9b55f4c5ab6c94d295b302394 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:42:52.190949 20187 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8f896f9b55f4c5ab6c94d295b302394 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f896f9b55f4c5ab6c94d295b302394" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40147 } }
I20260926 11:42:52.191303 20187 leader_election.cc:302] T 00000000000000000000000000000000 P a8f896f9b55f4c5ab6c94d295b302394 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8f896f9b55f4c5ab6c94d295b302394; no voters:
I20260926 11:42:52.191623 20187 leader_election.cc:288] T 00000000000000000000000000000000 P a8f896f9b55f4c5ab6c94d295b302394 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:42:52.191716 20192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8f896f9b55f4c5ab6c94d295b302394 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:42:52.191910 20192 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8f896f9b55f4c5ab6c94d295b302394 [term 1 LEADER]: Becoming Leader. State: Replica: a8f896f9b55f4c5ab6c94d295b302394, State: Running, Role: LEADER
I20260926 11:42:52.192276 20192 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8f896f9b55f4c5ab6c94d295b302394 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f896f9b55f4c5ab6c94d295b302394" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40147 } }
I20260926 11:42:52.193033 20187 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8f896f9b55f4c5ab6c94d295b302394 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:42:52.194126 20193 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8f896f9b55f4c5ab6c94d295b302394 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8f896f9b55f4c5ab6c94d295b302394. Latest consensus state: current_term: 1 leader_uuid: "a8f896f9b55f4c5ab6c94d295b302394" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f896f9b55f4c5ab6c94d295b302394" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40147 } } }
I20260926 11:42:52.194129 20194 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8f896f9b55f4c5ab6c94d295b302394 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8f896f9b55f4c5ab6c94d295b302394" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f896f9b55f4c5ab6c94d295b302394" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40147 } } }
I20260926 11:42:52.194350 20194 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8f896f9b55f4c5ab6c94d295b302394 [sys.catalog]: This master's current role is: LEADER
I20260926 11:42:52.194567 20190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:52.181610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42172 (local address 127.6.96.62:40147)
0926 11:42:52.182046 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:52.182064 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:52.182099 (+ 35us) server_negotiation.cc:407] Connection header received
0926 11:42:52.182785 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:52.182808 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:52.183203 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:52.183530 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:52.184324 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:52.185338 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:52.186209 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:52.186695 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:52.186867 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:52.189365 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:52.189371 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:52.189384 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:52.189414 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:52.191214 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:52.191716 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:52.191722 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:52.191729 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:52.191876 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:52.192206 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:52.192209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:52.192211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:52.192679 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:52.192857 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:52.193179 (+ 322us) server_negotiation.cc:299] Negotiation successful
0926 11:42:52.193510 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":134,"threads_started":1}
I20260926 11:42:52.194799 20201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:42:52.194342 20193 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8f896f9b55f4c5ab6c94d295b302394 [sys.catalog]: This master's current role is: LEADER
I20260926 11:42:52.197789 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 11:42:52.198057 20201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:42:52.203347 20201 catalog_manager.cc:1347] Generated new cluster ID: d5c801b960dd4cc5824fb1d0d6047081
I20260926 11:42:52.203428 20201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:42:52.213014 20201 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:42:52.214040 20201 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:42:52.219297 20201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8f896f9b55f4c5ab6c94d295b302394: Generated new TSK 0
I20260926 11:42:52.219980 20201 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:42:52.253800 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 20119
2026-09-26T11:42:52Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1322 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:42:52 dist-test-slave-xxxx krb5kdc[20214](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:42:52 dist-test-slave-xxxx krb5kdc[20214](info): set up 2 sockets
Sep 26 11:42:52 dist-test-slave-xxxx krb5kdc[20214](info): commencing operation
krb5kdc: starting...
W20260926 11:42:53.210779 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:42:53 dist-test-slave-xxxx krb5kdc[20214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:42:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:42:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:42:53.330999 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:46729
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:38355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:46729
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:42:53.440649 20227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:42:53.440920 20227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:42:53.440975 20227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:42:53.445227 20227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:42:53.445328 20227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:42:53.445361 20227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:42:53.445382 20227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:42:53.445405 20227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:42:53.451215 20227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:38355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:46729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:46729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:42:53.452298 20227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:42:53.453328 20227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 11:42:53.461756 20227 server_base.cc:1034] running on GCE node
W20260926 11:42:53.461652 20233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:53.461655 20232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:53.461722 20235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:53.462505 20227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:42:53.463637 20227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:42:53.464826 20227 hybrid_clock.cc:648] HybridClock initialized: now 1790422973464794 us; error 51 us; skew 500 ppm
Sep 26 11:42:53 dist-test-slave-xxxx krb5kdc[20214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422973, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:53.472711 20227 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:42:53.473654 20227 webserver.cc:460] Webserver started at http://127.6.96.62:37399/ using document root <none> and password file <none>
I20260926 11:42:53.474218 20227 fs_manager.cc:359] Metadata directory not provided
I20260926 11:42:53.474295 20227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:42:53.474498 20227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:42:53.476696 20227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "186821d926ea4008ad4cf4b2fb57f7d1"
format_stamp: "Formatted at 2026-09-26 11:42:53 on dist-test-slave-xxxx"
server_key: "ac5760225df9fa69cc5c15fa0d6411b3"
server_key_iv: "25d1991488ef25ca3bc8f92a38163343"
server_key_version: "encryptionkey@0"
I20260926 11:42:53.477257 20227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "186821d926ea4008ad4cf4b2fb57f7d1"
format_stamp: "Formatted at 2026-09-26 11:42:53 on dist-test-slave-xxxx"
server_key: "ac5760225df9fa69cc5c15fa0d6411b3"
server_key_iv: "25d1991488ef25ca3bc8f92a38163343"
server_key_version: "encryptionkey@0"
I20260926 11:42:53.480995 20227 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 11:42:53.483505 20242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:53.484982 20227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:42:53.485117 20227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "186821d926ea4008ad4cf4b2fb57f7d1"
format_stamp: "Formatted at 2026-09-26 11:42:53 on dist-test-slave-xxxx"
server_key: "ac5760225df9fa69cc5c15fa0d6411b3"
server_key_iv: "25d1991488ef25ca3bc8f92a38163343"
server_key_version: "encryptionkey@0"
I20260926 11:42:53.485220 20227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:42:53.497156 20227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:42:53.500219 20227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:42:53.500401 20227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:42:53.506860 20227 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:46729
I20260926 11:42:53.507126 20294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:46729 every 8 connection(s)
I20260926 11:42:53.507860 20227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:42:53.510594 20295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:42:53.516141 20295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1: Bootstrap starting.
I20260926 11:42:53.517407 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 20227
I20260926 11:42:53.517547 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:42:53.517805 6528 external_mini_cluster.cc:1428] Setting key 867d4a0877d3d043e6763fd0274e3b99
I20260926 11:42:53.518764 20295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1: Neither blocks nor log segments found. Creating new log.
I20260926 11:42:53.519578 20295 log.cc:824] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1: Log is configured to *not* fsync() on all Append() calls
I20260926 11:42:53.521797 20295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1: No bootstrap required, opened a new log
I20260926 11:42:53.525774 20295 raft_consensus.cc:348] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186821d926ea4008ad4cf4b2fb57f7d1" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46729 } }
I20260926 11:42:53.525991 20295 raft_consensus.cc:374] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:42:53.526029 20295 raft_consensus.cc:729] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 186821d926ea4008ad4cf4b2fb57f7d1, State: Initialized, Role: FOLLOWER
I20260926 11:42:53.526460 20295 consensus_queue.cc:260] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186821d926ea4008ad4cf4b2fb57f7d1" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46729 } }
I20260926 11:42:53.526614 20295 raft_consensus.cc:388] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:42:53.526690 20295 raft_consensus.cc:482] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:42:53.526790 20295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 0 FOLLOWER]: Advancing to term 1
Sep 26 11:42:53 dist-test-slave-xxxx krb5kdc[20214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:42:53.527791 20295 raft_consensus.cc:504] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186821d926ea4008ad4cf4b2fb57f7d1" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46729 } }
I20260926 11:42:53.528152 20295 leader_election.cc:302] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 186821d926ea4008ad4cf4b2fb57f7d1; no voters:
I20260926 11:42:53.528514 20295 leader_election.cc:288] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:42:53.528609 20300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:42:53.528818 20300 raft_consensus.cc:686] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 1 LEADER]: Becoming Leader. State: Replica: 186821d926ea4008ad4cf4b2fb57f7d1, State: Running, Role: LEADER
I20260926 11:42:53.529289 20300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186821d926ea4008ad4cf4b2fb57f7d1" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46729 } }
I20260926 11:42:53.529856 20295 sys_catalog.cc:564] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:42:53.531062 20302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 186821d926ea4008ad4cf4b2fb57f7d1. Latest consensus state: current_term: 1 leader_uuid: "186821d926ea4008ad4cf4b2fb57f7d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186821d926ea4008ad4cf4b2fb57f7d1" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46729 } } }
I20260926 11:42:53.531252 20302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [sys.catalog]: This master's current role is: LEADER
I20260926 11:42:53.531633 20301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "186821d926ea4008ad4cf4b2fb57f7d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186821d926ea4008ad4cf4b2fb57f7d1" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46729 } } }
I20260926 11:42:53.531780 20301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [sys.catalog]: This master's current role is: LEADER
I20260926 11:42:53.533243 20298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:53.518986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52558 (local address 127.6.96.62:46729)
0926 11:42:53.519460 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:53.519480 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:53.519522 (+ 42us) server_negotiation.cc:407] Connection header received
0926 11:42:53.520320 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:53.520340 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:53.520748 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:53.521105 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:53.521765 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:53.523254 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:53.524101 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:53.524843 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:53.524998 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:53.527505 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:53.527512 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:53.527530 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:53.527562 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:53.529479 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:53.530001 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:53.530007 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:53.530013 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:53.530194 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:53.530513 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:53.530517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:53.530518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:53.531075 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:53.531288 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:53.532167 (+ 879us) server_negotiation.cc:299] Negotiation successful
0926 11:42:53.532487 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":133,"threads_started":1}
W20260926 11:42:53.535192 20315 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 11:42:53.535319 20315 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 11:42:53.535410 20311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:42:53.536110 20311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:42:53.541395 20311 catalog_manager.cc:1347] Generated new cluster ID: 03adb1b3c6554353b379584709357d6e
I20260926 11:42:53.541462 20311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:42:53.546316 20311 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:42:53.547540 20311 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:42:53.556110 20311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1: Generated new TSK 0
I20260926 11:42:53.556795 20311 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.96.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
I20260926 11:42:53.590323 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:46729
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:38355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:42:53.707399 20321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:42:53.707674 20321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:42:53.707719 20321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:42:53.711880 20321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:42:53.711952 20321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:42:53.712014 20321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:42:53.717495 20321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:38355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:46729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:42:53.718771 20321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:42:53.719904 20321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:42:53.729530 20329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:53.729545 20326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:53.729592 20327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:53.729792 20321 server_base.cc:1034] running on GCE node
I20260926 11:42:53.730340 20321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:42:53.731117 20321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:42:53.732277 20321 hybrid_clock.cc:648] HybridClock initialized: now 1790422973732251 us; error 39 us; skew 500 ppm
Sep 26 11:42:53 dist-test-slave-xxxx krb5kdc[20214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422973, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:53.735035 20321 init.cc:377] Logged in from keytab as kudu/127.6.96.1@KRBTEST.COM (short username kudu)
I20260926 11:42:53.735980 20321 webserver.cc:460] Webserver started at http://127.6.96.1:46021/ using document root <none> and password file <none>
I20260926 11:42:53.736519 20321 fs_manager.cc:359] Metadata directory not provided
I20260926 11:42:53.736575 20321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:42:53.736750 20321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:42:53.738847 20321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d"
format_stamp: "Formatted at 2026-09-26 11:42:53 on dist-test-slave-xxxx"
server_key: "faefad837f5a0837ab0beaab237b8438"
server_key_iv: "907f2dab93f5902aa3e505d13e98f14a"
server_key_version: "encryptionkey@0"
I20260926 11:42:53.739387 20321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d"
format_stamp: "Formatted at 2026-09-26 11:42:53 on dist-test-slave-xxxx"
server_key: "faefad837f5a0837ab0beaab237b8438"
server_key_iv: "907f2dab93f5902aa3e505d13e98f14a"
server_key_version: "encryptionkey@0"
I20260926 11:42:53.743286 20321 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 11:42:53.745944 20336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:53.747347 20321 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:42:53.747462 20321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d"
format_stamp: "Formatted at 2026-09-26 11:42:53 on dist-test-slave-xxxx"
server_key: "faefad837f5a0837ab0beaab237b8438"
server_key_iv: "907f2dab93f5902aa3e505d13e98f14a"
server_key_version: "encryptionkey@0"
I20260926 11:42:53.747578 20321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:42:53.769807 20321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:42:53.773123 20321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:42:53.773304 20321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:42:53.773934 20321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:42:53.774782 20321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:42:53.774860 20321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:53.774925 20321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:42:53.774953 20321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:53.783545 20321 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:40911
I20260926 11:42:53.783571 20449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:40911 every 8 connection(s)
I20260926 11:42:53.784500 20321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:42:53.786549 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 20321
I20260926 11:42:53.786667 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:42:53.786967 6528 external_mini_cluster.cc:1428] Setting key d0c587a95570221d8121c0810951ae12
Sep 26 11:42:53 dist-test-slave-xxxx krb5kdc[20214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422973, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.96.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.2@KRBTEST.COM" created.
I20260926 11:42:53.798731 20298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:53.786299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:57243 (local address 127.6.96.62:46729)
0926 11:42:53.786475 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:53.786480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:53.787486 (+ 1006us) server_negotiation.cc:407] Connection header received
0926 11:42:53.788827 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:53.788831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:53.788882 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:53.788989 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:53.790300 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:53.790878 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:53.792058 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:53.792330 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:53.792422 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:53.795325 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:53.795329 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:53.795331 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:53.795357 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:53.796619 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:53.797134 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:53.797138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:53.797139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:53.797260 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:53.797623 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:53.797626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:53.797627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:53.797824 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:53.797938 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:53.798474 (+ 536us) server_negotiation.cc:299] Negotiation successful
0926 11:42:53.798600 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 11:42:53.799892 20452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:53.786821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.1:57243)
0926 11:42:53.787334 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:53.787390 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:53.788613 (+ 1223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:53.789168 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:53.789175 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:53.789586 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:42:53.790142 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:53.790159 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:53.791005 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:53.791008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:53.791930 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:53.791941 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:53.792486 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:53.792490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:53.792669 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:53.793329 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:42:53.793346 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:42:53.795106 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:42:53.796743 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:53.796748 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:53.796762 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:53.797031 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:53.797379 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:53.797382 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:53.797384 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:53.797529 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:53.797927 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:42:53.797932 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:42:53.798236 (+ 304us) client_negotiation.cc:769] Sending connection context
0926 11:42:53.798425 (+ 189us) client_negotiation.cc:240] Negotiation successful
0926 11:42:53.798794 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":123,"threads_started":1}
I20260926 11:42:53.801131 20450 heartbeater.cc:347] Connected to a master server at 127.6.96.62:46729
I20260926 11:42:53.801383 20450 heartbeater.cc:464] Registering TS with master...
I20260926 11:42:53.801878 20450 heartbeater.cc:511] Master 127.6.96.62:46729 requested a full tablet report, sending...
I20260926 11:42:53.803581 20259 ts_manager.cc:194] Registered new tserver with Master: ddd7c46e0f0f4774be80ac96a09b2f3d (127.6.96.1:40911)
I20260926 11:42:53.805177 20259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.1@KRBTEST.COM'} at 127.6.96.1:57243
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
I20260926 11:42:53.814026 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:46729
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:38355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:42:53.923446 20455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:42:53.923696 20455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:42:53.923738 20455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:42:53.927851 20455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:42:53.927906 20455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:42:53.927968 20455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:42:53.933262 20455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:38355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:46729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:42:53.934269 20455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:42:53.935287 20455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:42:53.944272 20460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:53.944284 20461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:53.944530 20455 server_base.cc:1034] running on GCE node
W20260926 11:42:53.944597 20463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:53.944986 20455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:42:53.945588 20455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:42:53.946751 20455 hybrid_clock.cc:648] HybridClock initialized: now 1790422973946729 us; error 36 us; skew 500 ppm
Sep 26 11:42:53 dist-test-slave-xxxx krb5kdc[20214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422973, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:53.949316 20455 init.cc:377] Logged in from keytab as kudu/127.6.96.2@KRBTEST.COM (short username kudu)
I20260926 11:42:53.950194 20455 webserver.cc:460] Webserver started at http://127.6.96.2:34251/ using document root <none> and password file <none>
I20260926 11:42:53.950709 20455 fs_manager.cc:359] Metadata directory not provided
I20260926 11:42:53.950773 20455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:42:53.951001 20455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:42:53.953018 20455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "385584ce2fca4815b11220c5443198e7"
format_stamp: "Formatted at 2026-09-26 11:42:53 on dist-test-slave-xxxx"
server_key: "880130e14b3c26f794116ac3e75047c4"
server_key_iv: "01d1b4f3758cab23f679ef77c0ad71ce"
server_key_version: "encryptionkey@0"
I20260926 11:42:53.953547 20455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "385584ce2fca4815b11220c5443198e7"
format_stamp: "Formatted at 2026-09-26 11:42:53 on dist-test-slave-xxxx"
server_key: "880130e14b3c26f794116ac3e75047c4"
server_key_iv: "01d1b4f3758cab23f679ef77c0ad71ce"
server_key_version: "encryptionkey@0"
I20260926 11:42:53.957082 20455 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 11:42:53.959491 20470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:53.960740 20455 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:42:53.960855 20455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "385584ce2fca4815b11220c5443198e7"
format_stamp: "Formatted at 2026-09-26 11:42:53 on dist-test-slave-xxxx"
server_key: "880130e14b3c26f794116ac3e75047c4"
server_key_iv: "01d1b4f3758cab23f679ef77c0ad71ce"
server_key_version: "encryptionkey@0"
I20260926 11:42:53.960951 20455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:42:53.976097 20455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:42:53.978919 20455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:42:53.979094 20455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:42:53.979684 20455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:42:53.980492 20455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:42:53.980537 20455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:53.980590 20455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:42:53.980616 20455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:53.988897 20455 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:42979
I20260926 11:42:53.988929 20583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:42979 every 8 connection(s)
I20260926 11:42:53.989822 20455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
Sep 26 11:42:53 dist-test-slave-xxxx krb5kdc[20214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422973, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:42:54.000001 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 20455
I20260926 11:42:54.000138 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:42:54.000415 6528 external_mini_cluster.cc:1428] Setting key a22b1acb61160cddbe3b40e9cd7a6dee
I20260926 11:42:54.002795 20298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:53.991545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:38719 (local address 127.6.96.62:46729)
0926 11:42:53.991666 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:53.991669 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:53.992512 (+ 843us) server_negotiation.cc:407] Connection header received
0926 11:42:53.993770 (+ 1258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:53.993773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:53.993830 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:53.993899 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:53.995115 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:53.995637 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:53.996742 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:53.996955 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:53.997040 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:53.999346 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:53.999349 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:53.999351 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:53.999375 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:54.000646 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:54.001224 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:54.001227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:54.001229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:54.001314 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:54.001624 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:54.001627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:54.001628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:54.001844 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:54.001937 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:54.002562 (+ 625us) server_negotiation.cc:299] Negotiation successful
0926 11:42:54.002676 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 11:42:54.003962 20586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:53.991935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.2:38719)
0926 11:42:53.992369 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:53.992412 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:53.993541 (+ 1129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:53.994089 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:53.994096 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:53.994452 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:42:53.994982 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:53.994998 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:53.995753 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:53.995756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:53.996617 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:53.996626 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:53.997091 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:53.997096 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:53.997251 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:53.997755 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:42:53.997771 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:42:53.999182 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:42:54.000772 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:54.000777 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:54.000790 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:54.001085 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:54.001399 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:54.001402 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:54.001403 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:54.001542 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:54.001948 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:42:54.001955 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:42:54.002277 (+ 322us) client_negotiation.cc:769] Sending connection context
0926 11:42:54.002499 (+ 222us) client_negotiation.cc:240] Negotiation successful
0926 11:42:54.002887 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":99,"threads_started":1}
I20260926 11:42:54.005105 20584 heartbeater.cc:347] Connected to a master server at 127.6.96.62:46729
I20260926 11:42:54.005344 20584 heartbeater.cc:464] Registering TS with master...
I20260926 11:42:54.005825 20584 heartbeater.cc:511] Master 127.6.96.62:46729 requested a full tablet report, sending...
I20260926 11:42:54.006875 20259 ts_manager.cc:194] Registered new tserver with Master: 385584ce2fca4815b11220c5443198e7 (127.6.96.2:42979)
I20260926 11:42:54.007431 20259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.2@KRBTEST.COM'} at 127.6.96.2:38719
WARNING: no policy specified for kudu/127.6.96.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
I20260926 11:42:54.028139 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:46729
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:38355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:42:54.138592 20589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:42:54.138906 20589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:42:54.138955 20589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:42:54.143288 20589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:42:54.143355 20589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:42:54.143419 20589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:42:54.148833 20589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:38355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:46729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:42:54.149943 20589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:42:54.151007 20589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:42:54.160917 20595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:54.160914 20597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:54.161034 20589 server_base.cc:1034] running on GCE node
W20260926 11:42:54.160914 20594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:54.161620 20589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:42:54.162338 20589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:42:54.163533 20589 hybrid_clock.cc:648] HybridClock initialized: now 1790422974163514 us; error 30 us; skew 500 ppm
Sep 26 11:42:54 dist-test-slave-xxxx krb5kdc[20214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422974, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:54.166455 20589 init.cc:377] Logged in from keytab as kudu/127.6.96.3@KRBTEST.COM (short username kudu)
I20260926 11:42:54.167434 20589 webserver.cc:460] Webserver started at http://127.6.96.3:39795/ using document root <none> and password file <none>
I20260926 11:42:54.167986 20589 fs_manager.cc:359] Metadata directory not provided
I20260926 11:42:54.168048 20589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:42:54.168224 20589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:42:54.170306 20589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "bf1fd9cc5e054d768f61e37e68fbad76"
format_stamp: "Formatted at 2026-09-26 11:42:54 on dist-test-slave-xxxx"
server_key: "ee4cc5aab0a38d55d4aaf75d7b0f68ea"
server_key_iv: "b246bfc2ca05d5726c835fb414f9501f"
server_key_version: "encryptionkey@0"
I20260926 11:42:54.170863 20589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "bf1fd9cc5e054d768f61e37e68fbad76"
format_stamp: "Formatted at 2026-09-26 11:42:54 on dist-test-slave-xxxx"
server_key: "ee4cc5aab0a38d55d4aaf75d7b0f68ea"
server_key_iv: "b246bfc2ca05d5726c835fb414f9501f"
server_key_version: "encryptionkey@0"
I20260926 11:42:54.174701 20589 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260926 11:42:54.177196 20604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:54.178447 20589 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:42:54.178553 20589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "bf1fd9cc5e054d768f61e37e68fbad76"
format_stamp: "Formatted at 2026-09-26 11:42:54 on dist-test-slave-xxxx"
server_key: "ee4cc5aab0a38d55d4aaf75d7b0f68ea"
server_key_iv: "b246bfc2ca05d5726c835fb414f9501f"
server_key_version: "encryptionkey@0"
I20260926 11:42:54.178647 20589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:42:54.200143 20589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:42:54.203245 20589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:42:54.203450 20589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:42:54.204025 20589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:42:54.204871 20589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:42:54.204919 20589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:54.204962 20589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:42:54.204991 20589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:54.213677 20589 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:35929
I20260926 11:42:54.213762 20717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:35929 every 8 connection(s)
I20260926 11:42:54.214584 20589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
Sep 26 11:42:54 dist-test-slave-xxxx krb5kdc[20214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422974, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:42:54.224582 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 20589
I20260926 11:42:54.224702 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:42:54.224947 6528 external_mini_cluster.cc:1428] Setting key c466ef809a89a77ffe80dd77512542c0
I20260926 11:42:54.228477 20298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:54.216422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:42353 (local address 127.6.96.62:46729)
0926 11:42:54.216563 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:54.216568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:54.217487 (+ 919us) server_negotiation.cc:407] Connection header received
0926 11:42:54.218762 (+ 1275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:54.218766 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:54.218851 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:54.218926 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:54.220134 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.220684 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.221854 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.222077 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.222164 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:54.224643 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:54.224646 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:54.224648 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:54.224674 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:54.226252 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:54.226904 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:54.226908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:54.226910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:54.226988 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:54.227374 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:54.227378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:54.227380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:54.227572 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:54.227661 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:54.228176 (+ 515us) server_negotiation.cc:299] Negotiation successful
0926 11:42:54.228305 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 11:42:54.229377 20720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:54.216793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.3:42353)
0926 11:42:54.217304 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:54.217351 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:54.218522 (+ 1171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:54.219108 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:54.219115 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:54.219486 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:42:54.219982 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:54.219996 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.220815 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.220818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:54.221728 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:54.221738 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.222230 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.222234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:54.222393 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:54.222971 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:42:54.222987 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:42:54.224473 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:42:54.226376 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:54.226382 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:54.226397 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:54.226766 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:54.227075 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:54.227078 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:54.227080 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:54.227276 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:54.227673 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:42:54.227680 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:42:54.227942 (+ 262us) client_negotiation.cc:769] Sending connection context
0926 11:42:54.228119 (+ 177us) client_negotiation.cc:240] Negotiation successful
0926 11:42:54.228514 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":108,"threads_started":1}
I20260926 11:42:54.230468 20718 heartbeater.cc:347] Connected to a master server at 127.6.96.62:46729
I20260926 11:42:54.230756 20718 heartbeater.cc:464] Registering TS with master...
I20260926 11:42:54.231278 20718 heartbeater.cc:511] Master 127.6.96.62:46729 requested a full tablet report, sending...
I20260926 11:42:54.232364 20259 ts_manager.cc:194] Registered new tserver with Master: bf1fd9cc5e054d768f61e37e68fbad76 (127.6.96.3:35929)
I20260926 11:42:54.232935 20259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.3@KRBTEST.COM'} at 127.6.96.3:42353
I20260926 11:42:54.239225 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 11:42:54.247946 20298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:54.241142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52574 (local address 127.6.96.62:46729)
0926 11:42:54.241296 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:54.241301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:54.241452 (+ 151us) server_negotiation.cc:407] Connection header received
0926 11:42:54.241561 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:54.241563 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:54.241604 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:54.241665 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:54.242254 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.242778 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.243621 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.243836 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.243917 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:54.245105 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:54.245108 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:54.245110 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:54.245134 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:54.246382 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:54.246763 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:54.246765 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:54.246766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:54.246887 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:54.247157 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:54.247159 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:54.247160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:54.247410 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:54.247514 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:54.247695 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 11:42:54.247794 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 11:42:54.250737 20259 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52574:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 11:42:54.253835 20259 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:42:54.267925 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:54.261982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:42979 (local address 127.0.0.1:45446)
0926 11:42:54.262852 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:54.262865 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:54.262944 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:54.263961 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:54.263964 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:54.263981 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:42:54.264113 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:54.264120 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.265427 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.265431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:54.266371 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:54.266382 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.267533 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.267536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:54.267617 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:54.267638 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 11:42:54.267688 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 11:42:54.267745 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":790,"mutex_wait_us":13,"thread_start_us":108,"threads_started":1}
I20260926 11:42:54.267957 20733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:54.262671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45446 (local address 127.6.96.2:42979)
0926 11:42:54.263398 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:54.263413 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:54.263434 (+ 21us) server_negotiation.cc:407] Connection header received
0926 11:42:54.263502 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:54.263507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:54.263686 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:54.263822 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:42:54.264293 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.265302 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.266629 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.267420 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.267530 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:54.267675 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:54.267726 (+ 51us) server_negotiation.cc:299] Negotiation successful
0926 11:42:54.267814 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":113,"threads_started":1}
I20260926 11:42:54.268220 20730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:54.261987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36424 (local address 127.6.96.1:40911)
0926 11:42:54.262805 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:54.262819 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:54.262873 (+ 54us) server_negotiation.cc:407] Connection header received
0926 11:42:54.262948 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:54.262954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:54.263166 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:54.263316 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:42:54.263670 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.264783 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.266249 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.267070 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.267168 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:54.267324 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:54.267988 (+ 664us) server_negotiation.cc:299] Negotiation successful
0926 11:42:54.268093 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":86,"threads_started":1}
I20260926 11:42:54.268147 20731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:54.262176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:35929 (local address 127.0.0.1:52154)
0926 11:42:54.263065 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:54.263077 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:54.263172 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:54.264369 (+ 1197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:54.264372 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:54.264390 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:42:54.264471 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:54.264476 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.265803 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.265806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:54.266784 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:54.266794 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.267811 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.267814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:54.267897 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:54.267918 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 11:42:54.267964 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 11:42:54.268030 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":833,"thread_start_us":88,"threads_started":1}
I20260926 11:42:54.267925 20729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:54.261874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:40911 (local address 127.0.0.1:36424)
0926 11:42:54.262441 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:54.262490 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:54.262619 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:54.263350 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:54.263356 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:54.263382 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:42:54.263527 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:54.263534 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.264912 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.264916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:54.266123 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:54.266132 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.267378 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.267380 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:54.267459 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:54.267493 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 11:42:54.267616 (+ 123us) client_negotiation.cc:240] Negotiation successful
0926 11:42:54.267745 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":5,"thread_start_us":70,"threads_started":1}
I20260926 11:42:54.268787 20734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:54.262675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52154 (local address 127.6.96.3:35929)
0926 11:42:54.263903 (+ 1228us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:54.263969 (+ 66us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:54.263993 (+ 24us) server_negotiation.cc:407] Connection header received
0926 11:42:54.264060 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:54.264065 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:54.264225 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:54.264593 (+ 368us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:42:54.264730 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.265662 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.266934 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.267688 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.268241 (+ 553us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:54.268374 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:54.268420 (+ 46us) server_negotiation.cc:299] Negotiation successful
0926 11:42:54.268653 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1119,"thread_start_us":1006,"threads_started":1}
I20260926 11:42:54.271420 20518 tablet_service.cc:1467] Processing CreateTablet for tablet 6570e34d01b44546bd2811636e9adfaa (DEFAULT_TABLE table=test-table [id=a47ad47be33748278f227818fe3584d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:42:54.271427 20652 tablet_service.cc:1467] Processing CreateTablet for tablet 6570e34d01b44546bd2811636e9adfaa (DEFAULT_TABLE table=test-table [id=a47ad47be33748278f227818fe3584d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:42:54.271430 20384 tablet_service.cc:1467] Processing CreateTablet for tablet 6570e34d01b44546bd2811636e9adfaa (DEFAULT_TABLE table=test-table [id=a47ad47be33748278f227818fe3584d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:42:54.272410 20518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6570e34d01b44546bd2811636e9adfaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:42:54.272414 20384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6570e34d01b44546bd2811636e9adfaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:42:54.272423 20652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6570e34d01b44546bd2811636e9adfaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:42:54.279596 20735 tablet_bootstrap.cc:492] T 6570e34d01b44546bd2811636e9adfaa P 385584ce2fca4815b11220c5443198e7: Bootstrap starting.
I20260926 11:42:54.280432 20736 tablet_bootstrap.cc:492] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76: Bootstrap starting.
I20260926 11:42:54.280889 20737 tablet_bootstrap.cc:492] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d: Bootstrap starting.
I20260926 11:42:54.282552 20736 tablet_bootstrap.cc:654] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76: Neither blocks nor log segments found. Creating new log.
I20260926 11:42:54.282727 20737 tablet_bootstrap.cc:654] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d: Neither blocks nor log segments found. Creating new log.
I20260926 11:42:54.282972 20735 tablet_bootstrap.cc:654] T 6570e34d01b44546bd2811636e9adfaa P 385584ce2fca4815b11220c5443198e7: Neither blocks nor log segments found. Creating new log.
I20260926 11:42:54.283839 20737 log.cc:824] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d: Log is configured to *not* fsync() on all Append() calls
I20260926 11:42:54.283835 20736 log.cc:824] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76: Log is configured to *not* fsync() on all Append() calls
I20260926 11:42:54.283835 20735 log.cc:824] T 6570e34d01b44546bd2811636e9adfaa P 385584ce2fca4815b11220c5443198e7: Log is configured to *not* fsync() on all Append() calls
I20260926 11:42:54.285876 20737 tablet_bootstrap.cc:492] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d: No bootstrap required, opened a new log
I20260926 11:42:54.285952 20736 tablet_bootstrap.cc:492] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76: No bootstrap required, opened a new log
I20260926 11:42:54.286083 20737 ts_tablet_manager.cc:1397] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 11:42:54.286149 20736 ts_tablet_manager.cc:1397] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260926 11:42:54.286255 20735 tablet_bootstrap.cc:492] T 6570e34d01b44546bd2811636e9adfaa P 385584ce2fca4815b11220c5443198e7: No bootstrap required, opened a new log
I20260926 11:42:54.286429 20735 ts_tablet_manager.cc:1397] T 6570e34d01b44546bd2811636e9adfaa P 385584ce2fca4815b11220c5443198e7: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260926 11:42:54.289172 20737 raft_consensus.cc:348] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } }
I20260926 11:42:54.289459 20737 raft_consensus.cc:374] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:42:54.289534 20737 raft_consensus.cc:729] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddd7c46e0f0f4774be80ac96a09b2f3d, State: Initialized, Role: FOLLOWER
I20260926 11:42:54.289996 20737 consensus_queue.cc:260] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } }
I20260926 11:42:54.290074 20735 raft_consensus.cc:348] T 6570e34d01b44546bd2811636e9adfaa P 385584ce2fca4815b11220c5443198e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } }
I20260926 11:42:54.290074 20736 raft_consensus.cc:348] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } }
I20260926 11:42:54.290429 20736 raft_consensus.cc:374] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:42:54.290444 20735 raft_consensus.cc:374] T 6570e34d01b44546bd2811636e9adfaa P 385584ce2fca4815b11220c5443198e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:42:54.290493 20735 raft_consensus.cc:729] T 6570e34d01b44546bd2811636e9adfaa P 385584ce2fca4815b11220c5443198e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 385584ce2fca4815b11220c5443198e7, State: Initialized, Role: FOLLOWER
I20260926 11:42:54.290530 20736 raft_consensus.cc:729] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf1fd9cc5e054d768f61e37e68fbad76, State: Initialized, Role: FOLLOWER
I20260926 11:42:54.290675 20450 heartbeater.cc:503] Master 127.6.96.62:46729 was elected leader, sending a full tablet report...
I20260926 11:42:54.291087 20737 ts_tablet_manager.cc:1428] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 11:42:54.291051 20736 consensus_queue.cc:260] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } }
I20260926 11:42:54.291051 20735 consensus_queue.cc:260] T 6570e34d01b44546bd2811636e9adfaa P 385584ce2fca4815b11220c5443198e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } }
I20260926 11:42:54.292022 20736 ts_tablet_manager.cc:1428] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 11:42:54.292407 20718 heartbeater.cc:503] Master 127.6.96.62:46729 was elected leader, sending a full tablet report...
I20260926 11:42:54.293411 20584 heartbeater.cc:503] Master 127.6.96.62:46729 was elected leader, sending a full tablet report...
I20260926 11:42:54.293972 20735 ts_tablet_manager.cc:1428] T 6570e34d01b44546bd2811636e9adfaa P 385584ce2fca4815b11220c5443198e7: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260926 11:42:54.412532 20743 raft_consensus.cc:482] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:42:54.412768 20743 raft_consensus.cc:504] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } }
I20260926 11:42:54.414191 20743 leader_election.cc:288] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ddd7c46e0f0f4774be80ac96a09b2f3d (127.6.96.1:40911), 385584ce2fca4815b11220c5443198e7 (127.6.96.2:42979)
I20260926 11:42:54.417788 20720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:54.414381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:40911 (local address 127.6.96.3:51247)
0926 11:42:54.414678 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:54.414692 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:54.414789 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:54.415039 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:54.415042 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:54.415057 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:42:54.415180 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:54.415185 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.415971 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.415974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:54.416756 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:54.416764 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.417384 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.417388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:54.417491 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:54.417513 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 11:42:54.417579 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 11:42:54.417651 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":4}
I20260926 11:42:54.417788 20730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:54.414478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:51247 (local address 127.6.96.1:40911)
0926 11:42:54.414591 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:54.414596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:54.414755 (+ 159us) server_negotiation.cc:407] Connection header received
0926 11:42:54.414903 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:54.414907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:54.414960 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:54.415023 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:42:54.415284 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.415867 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.416886 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.417277 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.417370 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:54.417433 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:54.417605 (+ 172us) server_negotiation.cc:299] Negotiation successful
0926 11:42:54.417670 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 11:42:54.417872 20733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:54.414546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:58665 (local address 127.6.96.2:42979)
0926 11:42:54.414679 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:54.414684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:54.414873 (+ 189us) server_negotiation.cc:407] Connection header received
0926 11:42:54.415011 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:54.415014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:54.415058 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:54.415119 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:42:54.415383 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.415962 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.416957 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.417372 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.417462 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:54.417519 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:54.417687 (+ 168us) server_negotiation.cc:299] Negotiation successful
0926 11:42:54.417759 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 11:42:54.417913 20744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:54.414463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:42979 (local address 127.6.96.3:58665)
0926 11:42:54.414776 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:54.414787 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:54.414914 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:54.415139 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:54.415141 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:54.415156 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:42:54.415277 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:54.415283 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.416067 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.416070 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:54.416843 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:54.416851 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.417483 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.417486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:54.417584 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:54.417600 (+ 16us) client_negotiation.cc:769] Sending connection context
0926 11:42:54.417661 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 11:42:54.417727 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":1,"thread_start_us":120,"threads_started":1}
I20260926 11:42:54.418577 20538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6570e34d01b44546bd2811636e9adfaa" candidate_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "385584ce2fca4815b11220c5443198e7" is_pre_election: true
I20260926 11:42:54.418723 20404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6570e34d01b44546bd2811636e9adfaa" candidate_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" is_pre_election: true
I20260926 11:42:54.418902 20538 raft_consensus.cc:2454] T 6570e34d01b44546bd2811636e9adfaa P 385584ce2fca4815b11220c5443198e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf1fd9cc5e054d768f61e37e68fbad76 in term 0.
I20260926 11:42:54.419023 20404 raft_consensus.cc:2454] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf1fd9cc5e054d768f61e37e68fbad76 in term 0.
I20260926 11:42:54.419343 20608 leader_election.cc:302] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 385584ce2fca4815b11220c5443198e7, bf1fd9cc5e054d768f61e37e68fbad76; no voters:
I20260926 11:42:54.419613 20743 raft_consensus.cc:2781] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:42:54.419696 20743 raft_consensus.cc:482] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:42:54.419728 20743 raft_consensus.cc:3037] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:42:54.420743 20743 raft_consensus.cc:504] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } }
I20260926 11:42:54.421141 20743 leader_election.cc:288] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76 [CANDIDATE]: Term 1 election: Requested vote from peers ddd7c46e0f0f4774be80ac96a09b2f3d (127.6.96.1:40911), 385584ce2fca4815b11220c5443198e7 (127.6.96.2:42979)
I20260926 11:42:54.421433 20404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6570e34d01b44546bd2811636e9adfaa" candidate_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d"
I20260926 11:42:54.421495 20538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6570e34d01b44546bd2811636e9adfaa" candidate_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "385584ce2fca4815b11220c5443198e7"
I20260926 11:42:54.421561 20404 raft_consensus.cc:3037] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:42:54.421608 20538 raft_consensus.cc:3037] T 6570e34d01b44546bd2811636e9adfaa P 385584ce2fca4815b11220c5443198e7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:42:54.422809 20538 raft_consensus.cc:2454] T 6570e34d01b44546bd2811636e9adfaa P 385584ce2fca4815b11220c5443198e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf1fd9cc5e054d768f61e37e68fbad76 in term 1.
I20260926 11:42:54.422854 20404 raft_consensus.cc:2454] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf1fd9cc5e054d768f61e37e68fbad76 in term 1.
I20260926 11:42:54.423100 20608 leader_election.cc:302] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 385584ce2fca4815b11220c5443198e7, bf1fd9cc5e054d768f61e37e68fbad76; no voters:
I20260926 11:42:54.423265 20743 raft_consensus.cc:2781] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:42:54.423481 20743 raft_consensus.cc:686] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76 [term 1 LEADER]: Becoming Leader. State: Replica: bf1fd9cc5e054d768f61e37e68fbad76, State: Running, Role: LEADER
I20260926 11:42:54.423805 20743 consensus_queue.cc:237] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } }
I20260926 11:42:54.426769 20257 catalog_manager.cc:5495] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf1fd9cc5e054d768f61e37e68fbad76 (127.6.96.3). New cstate: current_term: 1 leader_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } health_report { overall_health: HEALTHY } } }
I20260926 11:42:54.459175 20734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:54.455485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52160 (local address 127.6.96.3:35929)
0926 11:42:54.455635 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:54.455639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:54.455655 (+ 16us) server_negotiation.cc:407] Connection header received
0926 11:42:54.455732 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:54.455735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:54.455790 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:54.455886 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:42:54.456130 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.456698 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.457597 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.457819 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.457907 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:54.458253 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:42:54.458679 (+ 426us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:42:54.458787 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:54.458944 (+ 157us) server_negotiation.cc:299] Negotiation successful
0926 11:42:54.459006 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 11:42:54.465250 20404 raft_consensus.cc:1261] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d [term 1 FOLLOWER]: Refusing update from remote peer bf1fd9cc5e054d768f61e37e68fbad76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:42:54.465250 20538 raft_consensus.cc:1261] T 6570e34d01b44546bd2811636e9adfaa P 385584ce2fca4815b11220c5443198e7 [term 1 FOLLOWER]: Refusing update from remote peer bf1fd9cc5e054d768f61e37e68fbad76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260926 11:42:54.465538 20719 tablet.cc:2369] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:42:54.466104 20745 consensus_queue.cc:1035] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:42:54.466300 20743 consensus_queue.cc:1035] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:42:54.475217 20750 mvcc.cc:205] Tried to move back new op lower bound from 7333572503401582592 to 7333572503242485760. Current Snapshot: MvccSnapshot[applied={T|T < 7333572503401582592}]
I20260926 11:42:54.476392 20748 mvcc.cc:205] Tried to move back new op lower bound from 7333572503401582592 to 7333572503242485760. Current Snapshot: MvccSnapshot[applied={T|T < 7333572503401582592 or (T in {7333572503401582592})}]
I20260926 11:42:54.482776 20257 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:42:54.482964 20257 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:42:54.485455 20257 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1: Sending DeleteTablet for 3 replicas of tablet 6570e34d01b44546bd2811636e9adfaa
I20260926 11:42:54.486238 20518 tablet_service.cc:1514] Processing DeleteTablet for tablet 6570e34d01b44546bd2811636e9adfaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:42:54 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:45446
I20260926 11:42:54.486274 20384 tablet_service.cc:1514] Processing DeleteTablet for tablet 6570e34d01b44546bd2811636e9adfaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:42:54 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:36424
I20260926 11:42:54.486789 20758 tablet_replica.cc:330] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d: stopping tablet replica
I20260926 11:42:54.487013 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 20321
I20260926 11:42:54.487128 20652 tablet_service.cc:1514] Processing DeleteTablet for tablet 6570e34d01b44546bd2811636e9adfaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:42:54 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:52154
I20260926 11:42:54.487886 20759 tablet_replica.cc:330] T 6570e34d01b44546bd2811636e9adfaa P 385584ce2fca4815b11220c5443198e7: stopping tablet replica
I20260926 11:42:54.488080 20760 tablet_replica.cc:330] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76: stopping tablet replica
I20260926 11:42:54.488204 20759 raft_consensus.cc:2229] T 6570e34d01b44546bd2811636e9adfaa P 385584ce2fca4815b11220c5443198e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:42:54.488385 20760 raft_consensus.cc:2229] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76 [term 1 LEADER]: Raft consensus shutting down.
I20260926 11:42:54.488521 20759 raft_consensus.cc:2258] T 6570e34d01b44546bd2811636e9adfaa P 385584ce2fca4815b11220c5443198e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:42:54.488780 20760 raft_consensus.cc:2258] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:42:54.490211 20760 ts_tablet_manager.cc:1905] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:42:54.490214 20759 ts_tablet_manager.cc:1905] T 6570e34d01b44546bd2811636e9adfaa P 385584ce2fca4815b11220c5443198e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:42:54.492900 20760 ts_tablet_manager.cc:1918] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:42:54.492941 20759 ts_tablet_manager.cc:1918] T 6570e34d01b44546bd2811636e9adfaa P 385584ce2fca4815b11220c5443198e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:42:54.493064 20759 log.cc:1196] T 6570e34d01b44546bd2811636e9adfaa P 385584ce2fca4815b11220c5443198e7: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/wal/wals/6570e34d01b44546bd2811636e9adfaa
I20260926 11:42:54.493098 20760 log.cc:1196] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/6570e34d01b44546bd2811636e9adfaa
I20260926 11:42:54.493414 20759 ts_tablet_manager.cc:1939] T 6570e34d01b44546bd2811636e9adfaa P 385584ce2fca4815b11220c5443198e7: Deleting consensus metadata
I20260926 11:42:54.493501 20760 ts_tablet_manager.cc:1939] T 6570e34d01b44546bd2811636e9adfaa P bf1fd9cc5e054d768f61e37e68fbad76: Deleting consensus metadata
I20260926 11:42:54.494714 20246 catalog_manager.cc:4841] TS 385584ce2fca4815b11220c5443198e7 (127.6.96.2:42979): tablet 6570e34d01b44546bd2811636e9adfaa (table test-table [id=a47ad47be33748278f227818fe3584d2]) successfully deleted
I20260926 11:42:54.494977 20245 catalog_manager.cc:4841] TS bf1fd9cc5e054d768f61e37e68fbad76 (127.6.96.3:35929): tablet 6570e34d01b44546bd2811636e9adfaa (table test-table [id=a47ad47be33748278f227818fe3584d2]) successfully deleted
W20260926 11:42:54.495040 20244 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.96.1:40911 (error 108)
I20260926 11:42:54.495822 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 20455
I20260926 11:42:54.496026 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:54.495731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:40911 (local address 127.0.0.1:36440)
0926 11:42:54.495864 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:54.495944 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.1:40911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260926 11:42:54.496527 20244 catalog_manager.cc:4584] TS ddd7c46e0f0f4774be80ac96a09b2f3d (127.6.96.1:40911): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6570e34d01b44546bd2811636e9adfaa: Network error: Client connection negotiation failed: client connection to 127.6.96.1:40911: connect: Connection refused (error 111)
I20260926 11:42:54.501062 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 20589
I20260926 11:42:54.506387 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 20227
I20260926 11:42:54.511540 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:46729
--webserver_interface=127.6.96.62
--webserver_port=37399
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:38355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:46729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:42:54.619619 20762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:42:54.619884 20762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:42:54.619939 20762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:42:54.623878 20762 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:42:54.623934 20762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:42:54.623960 20762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:42:54.623982 20762 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:42:54.624010 20762 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:42:54.629285 20762 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:38355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:46729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:46729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=37399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:42:54.630309 20762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:42:54.631338 20762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:42:54.644098 20771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:54.644098 20768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:54.644403 20762 server_base.cc:1034] running on GCE node
W20260926 11:42:54.644098 20769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:54.645388 20762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:42:54.646616 20762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:42:54.647779 20762 hybrid_clock.cc:648] HybridClock initialized: now 1790422974647741 us; error 55 us; skew 500 ppm
Sep 26 11:42:54 dist-test-slave-xxxx krb5kdc[20214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422974, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:54.650621 20762 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:42:54.651618 20762 webserver.cc:460] Webserver started at http://127.6.96.62:37399/ using document root <none> and password file <none>
I20260926 11:42:54.652164 20762 fs_manager.cc:359] Metadata directory not provided
I20260926 11:42:54.652240 20762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:42:54.656126 20762 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 11:42:54.658327 20778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:54.659665 20762 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:42:54.659772 20762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "186821d926ea4008ad4cf4b2fb57f7d1"
format_stamp: "Formatted at 2026-09-26 11:42:53 on dist-test-slave-xxxx"
server_key: "ac5760225df9fa69cc5c15fa0d6411b3"
server_key_iv: "25d1991488ef25ca3bc8f92a38163343"
server_key_version: "encryptionkey@0"
I20260926 11:42:54.660373 20762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:42:54.673193 20762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:42:54.676419 20762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:42:54.676575 20762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:42:54.683555 20762 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:46729
I20260926 11:42:54.683840 20830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:46729 every 8 connection(s)
I20260926 11:42:54.684660 20762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:42:54.687289 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 20762
I20260926 11:42:54.687741 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:40911
--local_ip_for_outbound_sockets=127.6.96.1
--tserver_master_addrs=127.6.96.62:46729
--webserver_port=46021
--webserver_interface=127.6.96.1
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:38355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 11:42:54.689743 20831 sys_catalog.cc:263] Verifying existing consensus state
I20260926 11:42:54.691780 20831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1: Bootstrap starting.
I20260926 11:42:54.708850 20831 log.cc:824] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1: Log is configured to *not* fsync() on all Append() calls
I20260926 11:42:54.718129 20831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 11:42:54.718604 20831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1: Bootstrap complete.
I20260926 11:42:54.724264 20831 raft_consensus.cc:348] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186821d926ea4008ad4cf4b2fb57f7d1" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46729 } }
I20260926 11:42:54.724813 20831 raft_consensus.cc:729] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 186821d926ea4008ad4cf4b2fb57f7d1, State: Initialized, Role: FOLLOWER
I20260926 11:42:54.725378 20831 consensus_queue.cc:260] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186821d926ea4008ad4cf4b2fb57f7d1" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46729 } }
I20260926 11:42:54.725531 20831 raft_consensus.cc:388] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:42:54.725602 20831 raft_consensus.cc:482] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:42:54.725692 20831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 1 FOLLOWER]: Advancing to term 2
I20260926 11:42:54.727944 20831 raft_consensus.cc:504] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186821d926ea4008ad4cf4b2fb57f7d1" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46729 } }
I20260926 11:42:54.728289 20831 leader_election.cc:302] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 186821d926ea4008ad4cf4b2fb57f7d1; no voters:
I20260926 11:42:54.728609 20831 leader_election.cc:288] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 11:42:54.728734 20835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 11:42:54.728936 20835 raft_consensus.cc:686] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 2 LEADER]: Becoming Leader. State: Replica: 186821d926ea4008ad4cf4b2fb57f7d1, State: Running, Role: LEADER
I20260926 11:42:54.729255 20835 consensus_queue.cc:237] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186821d926ea4008ad4cf4b2fb57f7d1" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46729 } }
I20260926 11:42:54.729925 20831 sys_catalog.cc:564] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:42:54.730245 20835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 186821d926ea4008ad4cf4b2fb57f7d1. Latest consensus state: current_term: 2 leader_uuid: "186821d926ea4008ad4cf4b2fb57f7d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186821d926ea4008ad4cf4b2fb57f7d1" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46729 } } }
I20260926 11:42:54.730360 20835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [sys.catalog]: This master's current role is: LEADER
I20260926 11:42:54.730744 20836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "186821d926ea4008ad4cf4b2fb57f7d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186821d926ea4008ad4cf4b2fb57f7d1" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46729 } } }
I20260926 11:42:54.730893 20836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [sys.catalog]: This master's current role is: LEADER
I20260926 11:42:54.735118 20850 catalog_manager.cc:1259] Loaded cluster ID: 03adb1b3c6554353b379584709357d6e
I20260926 11:42:54.735186 20850 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1: loading cluster ID for follower catalog manager: success
W20260926 11:42:54.736167 20850 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260926 11:42:54.736241 20850 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260926 11:42:54.736296 20839 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:42:54.738742 20839 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:42:54.739313 20839 catalog_manager.cc:1259] Loaded cluster ID: 03adb1b3c6554353b379584709357d6e
I20260926 11:42:54.739379 20839 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:42:54.740507 20839 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260926 11:42:54.745340 20839 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260926 11:42:54.751508 20839 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:42:54.752422 20839 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1: Loaded TSK: 0
I20260926 11:42:54.753727 20839 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 11:42:54.807734 20833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:42:54.808020 20833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:42:54.808082 20833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:42:54.812255 20833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:42:54.812340 20833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:42:54.812422 20833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:42:54.818051 20833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:38355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:40911
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=46021
--tserver_master_addrs=127.6.96.62:46729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:42:54.819218 20833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:42:54.820274 20833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:42:54.829936 20861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:54.829923 20858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:54.829924 20859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:54.830315 20833 server_base.cc:1034] running on GCE node
I20260926 11:42:54.830767 20833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:42:54.831497 20833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:42:54.832659 20833 hybrid_clock.cc:648] HybridClock initialized: now 1790422974832645 us; error 48 us; skew 500 ppm
Sep 26 11:42:54 dist-test-slave-xxxx krb5kdc[20214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422974, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:54.835525 20833 init.cc:377] Logged in from keytab as kudu/127.6.96.1@KRBTEST.COM (short username kudu)
I20260926 11:42:54.836443 20833 webserver.cc:460] Webserver started at http://127.6.96.1:46021/ using document root <none> and password file <none>
I20260926 11:42:54.836998 20833 fs_manager.cc:359] Metadata directory not provided
I20260926 11:42:54.837077 20833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:42:54.841343 20833 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 11:42:54.843721 20868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:54.845110 20833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 11:42:54.845223 20833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d"
format_stamp: "Formatted at 2026-09-26 11:42:53 on dist-test-slave-xxxx"
server_key: "faefad837f5a0837ab0beaab237b8438"
server_key_iv: "907f2dab93f5902aa3e505d13e98f14a"
server_key_version: "encryptionkey@0"
I20260926 11:42:54.845781 20833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:42:54.879212 20833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:42:54.882277 20833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:42:54.882462 20833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:42:54.883092 20833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:42:54.884229 20875 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 11:42:54.888415 20833 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 11:42:54.888496 20833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260926 11:42:54.888562 20833 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 11:42:54.890400 20833 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 11:42:54.890450 20833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260926 11:42:54.890771 20875 tablet_bootstrap.cc:492] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d: Bootstrap starting.
I20260926 11:42:54.900223 20833 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:40911
I20260926 11:42:54.900311 20982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:40911 every 8 connection(s)
I20260926 11:42:54.901217 20833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:42:54.905256 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 20833
I20260926 11:42:54.905683 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:42979
--local_ip_for_outbound_sockets=127.6.96.2
--tserver_master_addrs=127.6.96.62:46729
--webserver_port=34251
--webserver_interface=127.6.96.2
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:38355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 11:42:54.909066 20875 log.cc:824] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d: Log is configured to *not* fsync() on all Append() calls
Sep 26 11:42:54 dist-test-slave-xxxx krb5kdc[20214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422974, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:42:54.917138 20875 tablet_bootstrap.cc:492] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 11:42:54.917572 20875 tablet_bootstrap.cc:492] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d: Bootstrap complete.
I20260926 11:42:54.918808 20875 ts_tablet_manager.cc:1397] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d: Time spent bootstrapping tablet: real 0.028s user 0.008s sys 0.016s
I20260926 11:42:54.919571 20986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:54.903172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.1:35529)
0926 11:42:54.903786 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:54.903839 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:54.904664 (+ 825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:54.906223 (+ 1559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:54.906233 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:54.906708 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:42:54.907350 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:54.907368 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.908644 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.908647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:54.909637 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:54.909648 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.910642 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.910646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:54.910883 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:54.911558 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:42:54.911578 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:42:54.913469 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:42:54.915953 (+ 2484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:54.915959 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:54.915974 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:54.916357 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:54.916802 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:54.916805 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:54.916807 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:54.916991 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:54.917787 (+ 796us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:42:54.917793 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:42:54.918085 (+ 292us) client_negotiation.cc:769] Sending connection context
0926 11:42:54.918262 (+ 177us) client_negotiation.cc:240] Negotiation successful
0926 11:42:54.918587 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":144,"threads_started":1}
I20260926 11:42:54.919961 20985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:54.903248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:35529 (local address 127.6.96.62:46729)
0926 11:42:54.903647 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:54.903667 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:54.903972 (+ 305us) server_negotiation.cc:407] Connection header received
0926 11:42:54.905147 (+ 1175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:54.905164 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:54.905727 (+ 563us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:54.906118 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:54.907521 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.908495 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.909773 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.910483 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.910664 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:54.913697 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:54.913703 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:54.913719 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:54.913752 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:54.915819 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:54.916521 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:54.916527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:54.916533 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:54.916686 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:54.917118 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:54.917122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:54.917123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:54.917603 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:54.917734 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:54.918432 (+ 698us) server_negotiation.cc:299] Negotiation successful
0926 11:42:54.918738 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":128,"threads_started":1}
I20260926 11:42:54.921566 20875 raft_consensus.cc:348] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } }
I20260926 11:42:54.921967 20875 raft_consensus.cc:729] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddd7c46e0f0f4774be80ac96a09b2f3d, State: Initialized, Role: FOLLOWER
I20260926 11:42:54.922410 20875 consensus_queue.cc:260] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } }
I20260926 11:42:54.922631 20983 heartbeater.cc:347] Connected to a master server at 127.6.96.62:46729
I20260926 11:42:54.922894 20983 heartbeater.cc:464] Registering TS with master...
I20260926 11:42:54.923408 20983 heartbeater.cc:511] Master 127.6.96.62:46729 requested a full tablet report, sending...
I20260926 11:42:54.923435 20875 ts_tablet_manager.cc:1428] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260926 11:42:54.925882 20795 ts_manager.cc:194] Registered new tserver with Master: ddd7c46e0f0f4774be80ac96a09b2f3d (127.6.96.1:40911)
I20260926 11:42:54.930289 20795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.1@KRBTEST.COM'} at 127.6.96.1:35529
I20260926 11:42:54.932974 20983 heartbeater.cc:503] Master 127.6.96.62:46729 was elected leader, sending a full tablet report...
Sep 26 11:42:54 dist-test-slave-xxxx krb5kdc[20214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422974, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for kudu/127.6.96.1@KRBTEST.COM
I20260926 11:42:54.939939 20990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:54.929185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:40911 (local address 127.0.0.1:36448)
0926 11:42:54.929556 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:54.929599 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:54.929721 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:54.930366 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:54.930373 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:54.930896 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:42:54.931010 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:54.931017 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.932284 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.932288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:54.933218 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:54.933231 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.933961 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.933979 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:54.934105 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:54.934668 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:42:54.934680 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:42:54.936289 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:42:54.938265 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:54.938273 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:54.938277 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:54.938503 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:54.938784 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:54.938786 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:54.938787 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:54.938896 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:54.939412 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:42:54.939416 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:42:54.939521 (+ 105us) client_negotiation.cc:769] Sending connection context
0926 11:42:54.939629 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 11:42:54.939779 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":118,"threads_started":1}
I20260926 11:42:54.940238 20991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:54.929260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36448 (local address 127.6.96.1:40911)
0926 11:42:54.929955 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:54.929963 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:54.929981 (+ 18us) server_negotiation.cc:407] Connection header received
0926 11:42:54.930037 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:54.930041 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:54.930230 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:54.930365 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:54.931124 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.932160 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.933359 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:54.933852 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:54.933955 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:54.936429 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:54.936439 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:54.936453 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:54.936480 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:54.938149 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:54.938603 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:54.938609 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:54.938611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:54.938705 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:54.938966 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:54.938969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:54.938971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:54.939313 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:54.939438 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:54.939801 (+ 363us) server_negotiation.cc:299] Negotiation successful
0926 11:42:54.939979 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":96,"threads_started":1}
I20260926 11:42:54.941193 20917 tablet_service.cc:1514] Processing DeleteTablet for tablet 6570e34d01b44546bd2811636e9adfaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:42:54 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:36448
I20260926 11:42:54.942004 20992 tablet_replica.cc:330] stopping tablet replica
I20260926 11:42:54.942286 20992 raft_consensus.cc:2229] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:42:54.942516 20992 raft_consensus.cc:2258] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:42:54.943459 20992 ts_tablet_manager.cc:1905] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:42:54.946935 20992 ts_tablet_manager.cc:1918] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:42:54.947018 20992 log.cc:1196] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/wal/wals/6570e34d01b44546bd2811636e9adfaa
I20260926 11:42:54.947242 20992 ts_tablet_manager.cc:1939] T 6570e34d01b44546bd2811636e9adfaa P ddd7c46e0f0f4774be80ac96a09b2f3d: Deleting consensus metadata
I20260926 11:42:54.948963 20780 catalog_manager.cc:4841] TS ddd7c46e0f0f4774be80ac96a09b2f3d (127.6.96.1:40911): tablet 6570e34d01b44546bd2811636e9adfaa (table test-table [id=a47ad47be33748278f227818fe3584d2]) successfully deleted
W20260926 11:42:55.025310 20987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:42:55.025596 20987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:42:55.025656 20987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:42:55.029832 20987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:42:55.029903 20987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:42:55.029986 20987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:42:55.035465 20987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:38355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:42979
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=34251
--tserver_master_addrs=127.6.96.62:46729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:42:55.036546 20987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:42:55.037607 20987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:42:55.047047 20997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:55.047057 21000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:55.047057 20998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:55.047096 20987 server_base.cc:1034] running on GCE node
I20260926 11:42:55.047636 20987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:42:55.048252 20987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:42:55.049437 20987 hybrid_clock.cc:648] HybridClock initialized: now 1790422975049415 us; error 32 us; skew 500 ppm
Sep 26 11:42:55 dist-test-slave-xxxx krb5kdc[20214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422975, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:55.052112 20987 init.cc:377] Logged in from keytab as kudu/127.6.96.2@KRBTEST.COM (short username kudu)
I20260926 11:42:55.052974 20987 webserver.cc:460] Webserver started at http://127.6.96.2:34251/ using document root <none> and password file <none>
I20260926 11:42:55.053508 20987 fs_manager.cc:359] Metadata directory not provided
I20260926 11:42:55.053584 20987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:42:55.057396 20987 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260926 11:42:55.059451 21007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:55.060765 20987 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:42:55.060877 20987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "385584ce2fca4815b11220c5443198e7"
format_stamp: "Formatted at 2026-09-26 11:42:53 on dist-test-slave-xxxx"
server_key: "880130e14b3c26f794116ac3e75047c4"
server_key_iv: "01d1b4f3758cab23f679ef77c0ad71ce"
server_key_version: "encryptionkey@0"
I20260926 11:42:55.061416 20987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:42:55.070657 20987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:42:55.073432 20987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:42:55.073602 20987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:42:55.074168 20987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:42:55.074993 20987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:42:55.075038 20987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:55.075091 20987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:42:55.075119 20987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:55.083288 20987 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:42979
I20260926 11:42:55.083408 21120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:42979 every 8 connection(s)
I20260926 11:42:55.084264 20987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
Sep 26 11:42:55 dist-test-slave-xxxx krb5kdc[20214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422975, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:42:55.093735 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 20987
I20260926 11:42:55.094209 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:35929
--local_ip_for_outbound_sockets=127.6.96.3
--tserver_master_addrs=127.6.96.62:46729
--webserver_port=39795
--webserver_interface=127.6.96.3
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:38355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 11:42:55.097680 20985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:55.085991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:59183 (local address 127.6.96.62:46729)
0926 11:42:55.086129 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:55.086134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:55.086940 (+ 806us) server_negotiation.cc:407] Connection header received
0926 11:42:55.088228 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:55.088232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:55.088291 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:55.088367 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:55.089580 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.090111 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.091267 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.091498 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.091587 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:55.094156 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:55.094176 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:55.094180 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:55.094210 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:55.095736 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:55.096239 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:55.096243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:55.096245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:55.096336 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:55.096615 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:55.096618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:55.096620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:55.096829 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:55.096917 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:55.097383 (+ 466us) server_negotiation.cc:299] Negotiation successful
0926 11:42:55.097513 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 11:42:55.098371 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:55.086328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.2:59183)
0926 11:42:55.086761 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:55.086803 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:55.088003 (+ 1200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:55.088550 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:55.088557 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:55.088909 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:42:55.089429 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:55.089444 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.090230 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.090233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:55.091143 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:55.091152 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.091633 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.091637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:55.091797 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:55.092343 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:42:55.092358 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:42:55.093924 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:42:55.095856 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:55.095861 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:55.095872 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:55.096111 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:55.096417 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:55.096419 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:55.096420 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:55.096540 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:55.096915 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:42:55.096920 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:42:55.097138 (+ 218us) client_negotiation.cc:769] Sending connection context
0926 11:42:55.097325 (+ 187us) client_negotiation.cc:240] Negotiation successful
0926 11:42:55.097571 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":104,"threads_started":1}
I20260926 11:42:55.099573 21121 heartbeater.cc:347] Connected to a master server at 127.6.96.62:46729
I20260926 11:42:55.099812 21121 heartbeater.cc:464] Registering TS with master...
I20260926 11:42:55.100281 21121 heartbeater.cc:511] Master 127.6.96.62:46729 requested a full tablet report, sending...
I20260926 11:42:55.101261 20795 ts_manager.cc:194] Registered new tserver with Master: 385584ce2fca4815b11220c5443198e7 (127.6.96.2:42979)
I20260926 11:42:55.101804 20795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.2@KRBTEST.COM'} at 127.6.96.2:59183
W20260926 11:42:55.202287 21124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:42:55.202555 21124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:42:55.202615 21124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:42:55.206645 21124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:42:55.206705 21124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:42:55.206782 21124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:42:55.211949 21124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:38355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:35929
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=39795
--tserver_master_addrs=127.6.96.62:46729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:42:55.212958 21124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:42:55.213939 21124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:42:55.223179 21130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:55.223161 21132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:55.223182 21129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:55.223308 21124 server_base.cc:1034] running on GCE node
I20260926 11:42:55.223857 21124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:42:55.224494 21124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:42:55.225651 21124 hybrid_clock.cc:648] HybridClock initialized: now 1790422975225632 us; error 30 us; skew 500 ppm
Sep 26 11:42:55 dist-test-slave-xxxx krb5kdc[20214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422975, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:55.228307 21124 init.cc:377] Logged in from keytab as kudu/127.6.96.3@KRBTEST.COM (short username kudu)
I20260926 11:42:55.229178 21124 webserver.cc:460] Webserver started at http://127.6.96.3:39795/ using document root <none> and password file <none>
I20260926 11:42:55.229735 21124 fs_manager.cc:359] Metadata directory not provided
I20260926 11:42:55.229807 21124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:42:55.233846 21124 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260926 11:42:55.235996 21139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:55.237335 21124 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 11:42:55.237438 21124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "bf1fd9cc5e054d768f61e37e68fbad76"
format_stamp: "Formatted at 2026-09-26 11:42:54 on dist-test-slave-xxxx"
server_key: "ee4cc5aab0a38d55d4aaf75d7b0f68ea"
server_key_iv: "b246bfc2ca05d5726c835fb414f9501f"
server_key_version: "encryptionkey@0"
I20260926 11:42:55.237991 21124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:42:55.269033 21124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:42:55.272277 21124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:42:55.272452 21124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:42:55.273052 21124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:42:55.273876 21124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:42:55.273922 21124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:55.273962 21124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:42:55.273988 21124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:55.283289 21124 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:35929
I20260926 11:42:55.283306 21252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:35929 every 8 connection(s)
I20260926 11:42:55.284394 21124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:42:55.290845 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 21124
Sep 26 11:42:55 dist-test-slave-xxxx krb5kdc[20214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422975, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:42:55.298223 20985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:55.286180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:47589 (local address 127.6.96.62:46729)
0926 11:42:55.286308 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:55.286312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:55.287206 (+ 894us) server_negotiation.cc:407] Connection header received
0926 11:42:55.288499 (+ 1293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:55.288503 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:55.288555 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:55.288632 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:55.289903 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.290455 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.291984 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.292214 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.292327 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:55.294726 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:55.294729 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:55.294731 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:55.294780 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:55.296190 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:55.296700 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:55.296703 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:55.296705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:55.296783 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:55.297115 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:55.297119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:55.297121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:55.297313 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:55.297398 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:55.297929 (+ 531us) server_negotiation.cc:299] Negotiation successful
0926 11:42:55.298053 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42,"spinlock_wait_cycles":14080}
I20260926 11:42:55.299149 21255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:55.286560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.3:47589)
0926 11:42:55.287040 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:55.287083 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:55.288272 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:55.288846 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:55.288853 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:55.289225 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:42:55.289716 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:55.289730 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.290610 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.290613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:55.291858 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:55.291869 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.292359 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.292363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:55.292543 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:55.293101 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:42:55.293118 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:42:55.294553 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:42:55.296307 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:55.296314 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:55.296334 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:55.296604 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:55.296864 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:55.296867 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:55.296870 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:55.297019 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:55.297411 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:42:55.297419 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:42:55.297686 (+ 267us) client_negotiation.cc:769] Sending connection context
0926 11:42:55.297872 (+ 186us) client_negotiation.cc:240] Negotiation successful
0926 11:42:55.298234 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":105,"threads_started":1}
I20260926 11:42:55.299518 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:55.291572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52590 (local address 127.6.96.62:46729)
0926 11:42:55.291832 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:55.291837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:55.291970 (+ 133us) server_negotiation.cc:407] Connection header received
0926 11:42:55.292095 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:55.292098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:55.292144 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:55.292258 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:55.292987 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.293487 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.294400 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.294624 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.294712 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:55.296237 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:55.296240 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:55.296242 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:55.296266 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:55.297570 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:55.297972 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:55.297975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:55.297976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:55.298047 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:55.298411 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:55.298414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:55.298416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:55.298594 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:55.298782 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:55.298992 (+ 210us) server_negotiation.cc:299] Negotiation successful
0926 11:42:55.299107 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"spinlock_wait_cycles":4992,"thread_start_us":98,"threads_started":1}
I20260926 11:42:55.300316 21253 heartbeater.cc:347] Connected to a master server at 127.6.96.62:46729
I20260926 11:42:55.300586 21253 heartbeater.cc:464] Registering TS with master...
I20260926 11:42:55.301112 21253 heartbeater.cc:511] Master 127.6.96.62:46729 requested a full tablet report, sending...
I20260926 11:42:55.302178 20794 ts_manager.cc:194] Registered new tserver with Master: bf1fd9cc5e054d768f61e37e68fbad76 (127.6.96.3:35929)
I20260926 11:42:55.302745 20794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.3@KRBTEST.COM'} at 127.6.96.3:47589
I20260926 11:42:55.313263 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 11:42:55.317637 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:55.313859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52596 (local address 127.6.96.62:46729)
0926 11:42:55.313987 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:55.313992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:55.314169 (+ 177us) server_negotiation.cc:407] Connection header received
0926 11:42:55.314306 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:55.314308 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:55.314413 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:55.314500 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:42:55.314672 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.315261 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.316089 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.316363 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.316442 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:55.316729 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:42:55.317151 (+ 422us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:42:55.317362 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:55.317438 (+ 76us) server_negotiation.cc:299] Negotiation successful
0926 11:42:55.317500 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 11:42:55.318284 20794 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:52596:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 11:42:55.321532 20794 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:42:55.329185 20917 tablet_service.cc:1467] Processing CreateTablet for tablet 2edfdc182fe64c958a55d8cd737f8e45 (DEFAULT_TABLE table=test-table [id=49d8a23938ab4ad183db419f43bece28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:42:55.329895 20917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2edfdc182fe64c958a55d8cd737f8e45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:42:55.332836 20875 tablet_bootstrap.cc:492] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d: Bootstrap starting.
I20260926 11:42:55.333777 20875 tablet_bootstrap.cc:654] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d: Neither blocks nor log segments found. Creating new log.
I20260926 11:42:55.333891 20990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:55.328168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:35929 (local address 127.0.0.1:52164)
0926 11:42:55.328284 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:55.328299 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:55.328384 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:55.329549 (+ 1165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:55.329552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:55.329569 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:42:55.329691 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:55.329698 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.331313 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.331317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:55.332485 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:55.332496 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.333524 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.333527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:55.333622 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:55.333641 (+ 19us) client_negotiation.cc:769] Sending connection context
0926 11:42:55.333682 (+ 41us) client_negotiation.cc:240] Negotiation successful
0926 11:42:55.333760 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":6}
I20260926 11:42:55.334013 21264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:55.328708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45462 (local address 127.6.96.2:42979)
0926 11:42:55.329060 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:55.329070 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:55.329245 (+ 175us) server_negotiation.cc:407] Connection header received
0926 11:42:55.329367 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:55.329372 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:55.329549 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:55.329698 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:42:55.329903 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.331256 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.332364 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.333112 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.333213 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:55.333360 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:55.333792 (+ 432us) server_negotiation.cc:299] Negotiation successful
0926 11:42:55.333888 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":161,"threads_started":1}
I20260926 11:42:55.334276 21262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:55.328515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:42979 (local address 127.0.0.1:45462)
0926 11:42:55.329152 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:55.329166 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:55.329267 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:55.329686 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:55.329689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:55.329708 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:42:55.329809 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:55.329815 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.331381 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.331383 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:55.332251 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:55.332260 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.333423 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.333427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:55.333514 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:55.333594 (+ 80us) client_negotiation.cc:769] Sending connection context
0926 11:42:55.333669 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 11:42:55.334153 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"mutex_wait_us":424,"thread_start_us":106,"threads_started":1}
I20260926 11:42:55.334623 21263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:55.328549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52164 (local address 127.6.96.3:35929)
0926 11:42:55.329015 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:55.329026 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:55.329049 (+ 23us) server_negotiation.cc:407] Connection header received
0926 11:42:55.329142 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:55.329147 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:55.329353 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:55.329825 (+ 472us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:42:55.329967 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.331141 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.332621 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.333411 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.334173 (+ 762us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:55.334324 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:55.334389 (+ 65us) server_negotiation.cc:299] Negotiation successful
0926 11:42:55.334492 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":109,"threads_started":1}
I20260926 11:42:55.335402 20875 tablet_bootstrap.cc:492] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d: No bootstrap required, opened a new log
I20260926 11:42:55.335497 20875 ts_tablet_manager.cc:1397] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 11:42:55.335913 20875 raft_consensus.cc:348] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } }
I20260926 11:42:55.336014 20875 raft_consensus.cc:374] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:42:55.336050 20875 raft_consensus.cc:729] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddd7c46e0f0f4774be80ac96a09b2f3d, State: Initialized, Role: FOLLOWER
I20260926 11:42:55.336200 20875 consensus_queue.cc:260] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } }
I20260926 11:42:55.336863 20875 ts_tablet_manager.cc:1428] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:42:55.337419 21055 tablet_service.cc:1467] Processing CreateTablet for tablet 2edfdc182fe64c958a55d8cd737f8e45 (DEFAULT_TABLE table=test-table [id=49d8a23938ab4ad183db419f43bece28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:42:55.337417 21187 tablet_service.cc:1467] Processing CreateTablet for tablet 2edfdc182fe64c958a55d8cd737f8e45 (DEFAULT_TABLE table=test-table [id=49d8a23938ab4ad183db419f43bece28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:42:55.338549 21055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2edfdc182fe64c958a55d8cd737f8e45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:42:55.338562 21187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2edfdc182fe64c958a55d8cd737f8e45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:42:55.347234 21267 tablet_bootstrap.cc:492] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76: Bootstrap starting.
I20260926 11:42:55.347535 21266 tablet_bootstrap.cc:492] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7: Bootstrap starting.
I20260926 11:42:55.349773 21267 tablet_bootstrap.cc:654] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76: Neither blocks nor log segments found. Creating new log.
I20260926 11:42:55.350178 21266 tablet_bootstrap.cc:654] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7: Neither blocks nor log segments found. Creating new log.
I20260926 11:42:55.351080 21266 log.cc:824] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7: Log is configured to *not* fsync() on all Append() calls
I20260926 11:42:55.351079 21267 log.cc:824] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76: Log is configured to *not* fsync() on all Append() calls
I20260926 11:42:55.353123 21267 tablet_bootstrap.cc:492] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76: No bootstrap required, opened a new log
I20260926 11:42:55.353226 21266 tablet_bootstrap.cc:492] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7: No bootstrap required, opened a new log
I20260926 11:42:55.353327 21267 ts_tablet_manager.cc:1397] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260926 11:42:55.353404 21266 ts_tablet_manager.cc:1397] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260926 11:42:55.356292 21267 raft_consensus.cc:348] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } }
I20260926 11:42:55.356292 21266 raft_consensus.cc:348] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } }
I20260926 11:42:55.356527 21266 raft_consensus.cc:374] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:42:55.356529 21267 raft_consensus.cc:374] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:42:55.356559 21266 raft_consensus.cc:729] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 385584ce2fca4815b11220c5443198e7, State: Initialized, Role: FOLLOWER
I20260926 11:42:55.356576 21267 raft_consensus.cc:729] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf1fd9cc5e054d768f61e37e68fbad76, State: Initialized, Role: FOLLOWER
I20260926 11:42:55.356956 21266 consensus_queue.cc:260] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } }
I20260926 11:42:55.356957 21267 consensus_queue.cc:260] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } }
I20260926 11:42:55.357601 21253 heartbeater.cc:503] Master 127.6.96.62:46729 was elected leader, sending a full tablet report...
I20260926 11:42:55.357934 21267 ts_tablet_manager.cc:1428] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260926 11:42:55.358213 21121 heartbeater.cc:503] Master 127.6.96.62:46729 was elected leader, sending a full tablet report...
I20260926 11:42:55.358783 21266 ts_tablet_manager.cc:1428] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260926 11:42:55.363175 21271 raft_consensus.cc:482] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:42:55.363348 21271 raft_consensus.cc:504] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } }
I20260926 11:42:55.364348 21271 leader_election.cc:288] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf1fd9cc5e054d768f61e37e68fbad76 (127.6.96.3:35929), ddd7c46e0f0f4774be80ac96a09b2f3d (127.6.96.1:40911)
I20260926 11:42:55.367998 21263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:55.364595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:43653 (local address 127.6.96.3:35929)
0926 11:42:55.364695 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:55.364700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:55.364830 (+ 130us) server_negotiation.cc:407] Connection header received
0926 11:42:55.364940 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:55.364943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:55.364988 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:55.365052 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:42:55.365434 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.366026 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.367088 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.367497 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.367588 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:55.367647 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:55.367816 (+ 169us) server_negotiation.cc:299] Negotiation successful
0926 11:42:55.367884 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 11:42:55.367998 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:55.364501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:35929 (local address 127.6.96.2:43653)
0926 11:42:55.364760 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:55.364774 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:55.364861 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:55.365147 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:55.365150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:55.365165 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:42:55.365277 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:55.365283 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.366131 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.366135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:55.366978 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:55.366988 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.367603 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.367606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:55.367709 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:55.367730 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 11:42:55.367800 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 11:42:55.367871 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":22}
I20260926 11:42:55.368359 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:55.364570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:40911 (local address 127.6.96.2:42835)
0926 11:42:55.364866 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:55.364879 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:55.364979 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:55.365596 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:55.365599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:55.365614 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:42:55.365741 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:55.365746 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.366462 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.366465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:55.367297 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:55.367307 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.367971 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.367975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:55.368069 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:55.368090 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 11:42:55.368172 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 11:42:55.368231 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":2,"thread_start_us":149,"threads_started":1}
I20260926 11:42:55.368599 21207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edfdc182fe64c958a55d8cd737f8e45" candidate_uuid: "385584ce2fca4815b11220c5443198e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" is_pre_election: true
I20260926 11:42:55.368845 21207 raft_consensus.cc:2454] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 385584ce2fca4815b11220c5443198e7 in term 0.
I20260926 11:42:55.369253 20991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:55.365087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:42835 (local address 127.6.96.1:40911)
0926 11:42:55.365220 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:55.365224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:55.365238 (+ 14us) server_negotiation.cc:407] Connection header received
0926 11:42:55.365440 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:55.365443 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:55.365501 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:55.365577 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:42:55.365835 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.366362 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.367405 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.367857 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.367952 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:55.368098 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:55.369050 (+ 952us) server_negotiation.cc:299] Negotiation successful
0926 11:42:55.369144 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 11:42:55.369251 21010 leader_election.cc:302] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 385584ce2fca4815b11220c5443198e7, bf1fd9cc5e054d768f61e37e68fbad76; no voters:
I20260926 11:42:55.369500 21271 raft_consensus.cc:2781] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:42:55.369592 21271 raft_consensus.cc:482] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:42:55.369639 21271 raft_consensus.cc:3037] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:42:55.369727 20937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edfdc182fe64c958a55d8cd737f8e45" candidate_uuid: "385584ce2fca4815b11220c5443198e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" is_pre_election: true
I20260926 11:42:55.369968 20937 raft_consensus.cc:2454] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 385584ce2fca4815b11220c5443198e7 in term 0.
I20260926 11:42:55.370731 21271 raft_consensus.cc:504] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } }
I20260926 11:42:55.371074 21271 leader_election.cc:288] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7 [CANDIDATE]: Term 1 election: Requested vote from peers bf1fd9cc5e054d768f61e37e68fbad76 (127.6.96.3:35929), ddd7c46e0f0f4774be80ac96a09b2f3d (127.6.96.1:40911)
I20260926 11:42:55.371417 20937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edfdc182fe64c958a55d8cd737f8e45" candidate_uuid: "385584ce2fca4815b11220c5443198e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d"
I20260926 11:42:55.371526 21207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edfdc182fe64c958a55d8cd737f8e45" candidate_uuid: "385584ce2fca4815b11220c5443198e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf1fd9cc5e054d768f61e37e68fbad76"
I20260926 11:42:55.371551 20937 raft_consensus.cc:3037] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:42:55.371654 21207 raft_consensus.cc:3037] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:42:55.372822 20937 raft_consensus.cc:2454] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 385584ce2fca4815b11220c5443198e7 in term 1.
I20260926 11:42:55.372821 21207 raft_consensus.cc:2454] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 385584ce2fca4815b11220c5443198e7 in term 1.
I20260926 11:42:55.373078 21010 leader_election.cc:302] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 385584ce2fca4815b11220c5443198e7, bf1fd9cc5e054d768f61e37e68fbad76; no voters:
I20260926 11:42:55.373435 21271 raft_consensus.cc:2781] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:42:55.373651 21271 raft_consensus.cc:686] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7 [term 1 LEADER]: Becoming Leader. State: Replica: 385584ce2fca4815b11220c5443198e7, State: Running, Role: LEADER
I20260926 11:42:55.374110 21271 consensus_queue.cc:237] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } }
I20260926 11:42:55.377095 20794 catalog_manager.cc:5495] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 385584ce2fca4815b11220c5443198e7 (127.6.96.2). New cstate: current_term: 1 leader_uuid: "385584ce2fca4815b11220c5443198e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } health_report { overall_health: UNKNOWN } } }
I20260926 11:42:55.391990 21264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:55.388776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45478 (local address 127.6.96.2:42979)
0926 11:42:55.388903 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:55.388907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:55.388919 (+ 12us) server_negotiation.cc:407] Connection header received
0926 11:42:55.388991 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:55.388993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:55.389047 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:55.389219 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:42:55.389349 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.389839 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.390641 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:55.390922 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:55.391008 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:55.391258 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:42:55.391674 (+ 416us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:42:55.391762 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:55.391808 (+ 46us) server_negotiation.cc:299] Negotiation successful
0926 11:42:55.391860 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 11:42:55.397807 21207 raft_consensus.cc:1261] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76 [term 1 FOLLOWER]: Refusing update from remote peer 385584ce2fca4815b11220c5443198e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:42:55.397805 20937 raft_consensus.cc:1261] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [term 1 FOLLOWER]: Refusing update from remote peer 385584ce2fca4815b11220c5443198e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:42:55.398444 21273 consensus_queue.cc:1035] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:42:55.398613 21271 consensus_queue.cc:1035] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 11:42:55.402478 20984 tablet.cc:2369] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:42:55.404269 21277 mvcc.cc:205] Tried to move back new op lower bound from 7333572507222134784 to 7333572507134816256. Current Snapshot: MvccSnapshot[applied={T|T < 7333572507222134784}]
I20260926 11:42:55.406365 21278 mvcc.cc:205] Tried to move back new op lower bound from 7333572507222134784 to 7333572507134816256. Current Snapshot: MvccSnapshot[applied={T|T < 7333572507222134784}]
I20260926 11:42:55.413107 20795 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:52596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:42:55.413255 20795 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:52596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:42:55.415493 20795 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1: Sending DeleteTablet for 3 replicas of tablet 2edfdc182fe64c958a55d8cd737f8e45
I20260926 11:42:55.416210 20917 tablet_service.cc:1514] Processing DeleteTablet for tablet 2edfdc182fe64c958a55d8cd737f8e45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:42:55 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:36448
I20260926 11:42:55.416178 21055 tablet_service.cc:1514] Processing DeleteTablet for tablet 2edfdc182fe64c958a55d8cd737f8e45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:42:55 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:45462
I20260926 11:42:55.416213 21187 tablet_service.cc:1514] Processing DeleteTablet for tablet 2edfdc182fe64c958a55d8cd737f8e45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:42:55 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:52164
I20260926 11:42:55.416404 20992 tablet_replica.cc:330] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d: stopping tablet replica
I20260926 11:42:55.416517 20992 raft_consensus.cc:2229] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:42:55.416664 20992 raft_consensus.cc:2258] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:42:55.416754 21285 tablet_replica.cc:330] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7: stopping tablet replica
I20260926 11:42:55.416932 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 20833
I20260926 11:42:55.417155 21285 raft_consensus.cc:2229] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7 [term 1 LEADER]: Raft consensus shutting down.
I20260926 11:42:55.417599 21285 raft_consensus.cc:2258] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:42:55.419224 21285 ts_tablet_manager.cc:1905] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:42:55.422075 21285 ts_tablet_manager.cc:1918] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:42:55.422178 21285 log.cc:1196] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/wal/wals/2edfdc182fe64c958a55d8cd737f8e45
I20260926 11:42:55.422504 21285 ts_tablet_manager.cc:1939] T 2edfdc182fe64c958a55d8cd737f8e45 P 385584ce2fca4815b11220c5443198e7: Deleting consensus metadata
W20260926 11:42:55.422942 20780 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.96.1:40911 (error 108)
I20260926 11:42:55.423249 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 20987
I20260926 11:42:55.424681 21286 tablet_replica.cc:330] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76: stopping tablet replica
I20260926 11:42:55.424858 21262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:55.424507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:40911 (local address 127.0.0.1:36456)
0926 11:42:55.424687 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:55.424757 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.1:40911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260926 11:42:55.425029 21286 raft_consensus.cc:2229] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:42:55.425384 21286 raft_consensus.cc:2258] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 11:42:55.425027 20780 catalog_manager.cc:4584] TS ddd7c46e0f0f4774be80ac96a09b2f3d (127.6.96.1:40911): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2edfdc182fe64c958a55d8cd737f8e45: Network error: Client connection negotiation failed: client connection to 127.6.96.1:40911: connect: Connection refused (error 111)
I20260926 11:42:55.426720 21286 ts_tablet_manager.cc:1905] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:42:55.429064 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 21124
I20260926 11:42:55.429083 21286 ts_tablet_manager.cc:1918] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:42:55.429195 21286 log.cc:1196] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/2edfdc182fe64c958a55d8cd737f8e45
I20260926 11:42:55.429456 21286 ts_tablet_manager.cc:1939] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76: Deleting consensus metadata
I20260926 11:42:55.430477 21262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:55.429291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:42979 (local address 127.0.0.1:45490)
0926 11:42:55.430355 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:55.430408 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.2:42979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":989}
I20260926 11:42:55.434159 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 20762
I20260926 11:42:55.439733 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:46729
--webserver_interface=127.6.96.62
--webserver_port=37399
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:38355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:46729
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:42:55.547899 21288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:42:55.548189 21288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:42:55.548229 21288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:42:55.552343 21288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:42:55.552414 21288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:42:55.552433 21288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:42:55.552448 21288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:42:55.552464 21288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:42:55.558002 21288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:38355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:46729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:46729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=37399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:42:55.559180 21288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:42:55.560222 21288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:42:55.568842 21296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:55.568878 21293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:55.568851 21294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:55.569092 21288 server_base.cc:1034] running on GCE node
I20260926 11:42:55.569834 21288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:42:55.570976 21288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:42:55.572146 21288 hybrid_clock.cc:648] HybridClock initialized: now 1790422975572127 us; error 29 us; skew 500 ppm
Sep 26 11:42:55 dist-test-slave-xxxx krb5kdc[20214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422975, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:55.575044 21288 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:42:55.576016 21288 webserver.cc:460] Webserver started at http://127.6.96.62:37399/ using document root <none> and password file <none>
I20260926 11:42:55.576558 21288 fs_manager.cc:359] Metadata directory not provided
I20260926 11:42:55.576627 21288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:42:55.580551 21288 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260926 11:42:55.582691 21303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:55.584026 21288 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:42:55.584144 21288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "186821d926ea4008ad4cf4b2fb57f7d1"
format_stamp: "Formatted at 2026-09-26 11:42:53 on dist-test-slave-xxxx"
server_key: "ac5760225df9fa69cc5c15fa0d6411b3"
server_key_iv: "25d1991488ef25ca3bc8f92a38163343"
server_key_version: "encryptionkey@0"
I20260926 11:42:55.584695 21288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:42:55.599624 21288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:42:55.602612 21288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:42:55.602790 21288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:42:55.609710 21288 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:46729
I20260926 11:42:55.609977 21355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:46729 every 8 connection(s)
I20260926 11:42:55.610750 21288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:42:55.615139 21356 sys_catalog.cc:263] Verifying existing consensus state
I20260926 11:42:55.616290 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 21288
I20260926 11:42:55.616714 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:40911
--local_ip_for_outbound_sockets=127.6.96.1
--tserver_master_addrs=127.6.96.62:46729
--webserver_port=46021
--webserver_interface=127.6.96.1
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:38355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 11:42:55.616863 21356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1: Bootstrap starting.
I20260926 11:42:55.634416 21356 log.cc:824] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1: Log is configured to *not* fsync() on all Append() calls
I20260926 11:42:55.645665 21356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 11:42:55.646026 21356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1: Bootstrap complete.
I20260926 11:42:55.649938 21356 raft_consensus.cc:348] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186821d926ea4008ad4cf4b2fb57f7d1" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46729 } }
I20260926 11:42:55.650350 21356 raft_consensus.cc:729] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 186821d926ea4008ad4cf4b2fb57f7d1, State: Initialized, Role: FOLLOWER
I20260926 11:42:55.650882 21356 consensus_queue.cc:260] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186821d926ea4008ad4cf4b2fb57f7d1" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46729 } }
I20260926 11:42:55.651012 21356 raft_consensus.cc:388] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:42:55.651082 21356 raft_consensus.cc:482] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:42:55.651177 21356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 2 FOLLOWER]: Advancing to term 3
I20260926 11:42:55.653119 21356 raft_consensus.cc:504] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186821d926ea4008ad4cf4b2fb57f7d1" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46729 } }
I20260926 11:42:55.653461 21356 leader_election.cc:302] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 186821d926ea4008ad4cf4b2fb57f7d1; no voters:
I20260926 11:42:55.653779 21356 leader_election.cc:288] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 11:42:55.653883 21360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 11:42:55.654076 21360 raft_consensus.cc:686] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [term 3 LEADER]: Becoming Leader. State: Replica: 186821d926ea4008ad4cf4b2fb57f7d1, State: Running, Role: LEADER
I20260926 11:42:55.654390 21360 consensus_queue.cc:237] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186821d926ea4008ad4cf4b2fb57f7d1" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46729 } }
I20260926 11:42:55.655081 21356 sys_catalog.cc:564] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:42:55.655903 21362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 186821d926ea4008ad4cf4b2fb57f7d1. Latest consensus state: current_term: 3 leader_uuid: "186821d926ea4008ad4cf4b2fb57f7d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186821d926ea4008ad4cf4b2fb57f7d1" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46729 } } }
I20260926 11:42:55.656029 21362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [sys.catalog]: This master's current role is: LEADER
I20260926 11:42:55.656297 21361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "186821d926ea4008ad4cf4b2fb57f7d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186821d926ea4008ad4cf4b2fb57f7d1" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46729 } } }
I20260926 11:42:55.656378 21361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1 [sys.catalog]: This master's current role is: LEADER
I20260926 11:42:55.660152 21375 catalog_manager.cc:1259] Loaded cluster ID: 03adb1b3c6554353b379584709357d6e
I20260926 11:42:55.660226 21375 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1: loading cluster ID for follower catalog manager: success
W20260926 11:42:55.661324 21375 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 186821d926ea4008ad4cf4b2fb57f7d1: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260926 11:42:55.661398 21375 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260926 11:42:55.661602 21369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:42:55.664367 21369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:42:55.664870 21369 catalog_manager.cc:1259] Loaded cluster ID: 03adb1b3c6554353b379584709357d6e
I20260926 11:42:55.664929 21369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260926 11:42:55.665536 21369 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f92c588a9ad google::LogMessage::Fail() at ??:0
@ 0x7f92c588d88a google::LogMessage::SendToLog() at ??:0
@ 0x7f92c588a4de google::LogMessage::Flush() at ??:0
@ 0x7f92c588ba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f92d01c9fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f92d01cad2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f92d01c7579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f92d020b640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f92c7adc3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f92c7adcc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f92c7ade3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f92c7acd512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f92c91116db start_thread at ??:0
@ 0x7f92c438861f clone at ??:0
W20260926 11:42:55.753154 21358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:42:55.753407 21358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:42:55.753448 21358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:42:55.757553 21358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:42:55.757614 21358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:42:55.757686 21358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:42:55.763017 21358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:38355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:40911
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=46021
--tserver_master_addrs=127.6.96.62:46729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:42:55.764119 21358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:42:55.765105 21358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:42:55.774470 21398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:55.774544 21400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:55.774544 21397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:55.775174 21358 server_base.cc:1034] running on GCE node
I20260926 11:42:55.775583 21358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:42:55.776232 21358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:42:55.777405 21358 hybrid_clock.cc:648] HybridClock initialized: now 1790422975777374 us; error 49 us; skew 500 ppm
Sep 26 11:42:55 dist-test-slave-xxxx krb5kdc[20214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422975, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:55.780385 21358 init.cc:377] Logged in from keytab as kudu/127.6.96.1@KRBTEST.COM (short username kudu)
I20260926 11:42:55.781569 21358 webserver.cc:460] Webserver started at http://127.6.96.1:46021/ using document root <none> and password file <none>
I20260926 11:42:55.782318 21358 fs_manager.cc:359] Metadata directory not provided
I20260926 11:42:55.782415 21358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:42:55.787708 21358 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 11:42:55.790565 21407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:55.792200 21358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:42:55.792339 21358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d"
format_stamp: "Formatted at 2026-09-26 11:42:53 on dist-test-slave-xxxx"
server_key: "faefad837f5a0837ab0beaab237b8438"
server_key_iv: "907f2dab93f5902aa3e505d13e98f14a"
server_key_version: "encryptionkey@0"
I20260926 11:42:55.792940 21358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:42:55.813460 21358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:42:55.817356 21358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:42:55.817562 21358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:42:55.818322 21358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:42:55.819828 21414 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 11:42:55.824391 21358 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 11:42:55.824486 21358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260926 11:42:55.824558 21358 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 11:42:55.826946 21358 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 11:42:55.827011 21358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260926 11:42:55.827298 21414 tablet_bootstrap.cc:492] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d: Bootstrap starting.
I20260926 11:42:55.837819 21358 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:40911
I20260926 11:42:55.837881 21521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:40911 every 8 connection(s)
I20260926 11:42:55.839074 21358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:42:55.843680 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 21358
I20260926 11:42:55.844183 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:42979
--local_ip_for_outbound_sockets=127.6.96.2
--tserver_master_addrs=127.6.96.62:46729
--webserver_port=34251
--webserver_interface=127.6.96.2
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:38355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 11:42:55.846632 21414 log.cc:824] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d: Log is configured to *not* fsync() on all Append() calls
I20260926 11:42:55.855329 21414 tablet_bootstrap.cc:492] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 11:42:55.855746 21414 tablet_bootstrap.cc:492] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d: Bootstrap complete.
I20260926 11:42:55.857043 21414 ts_tablet_manager.cc:1397] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d: Time spent bootstrapping tablet: real 0.030s user 0.015s sys 0.012s
I20260926 11:42:55.860178 21414 raft_consensus.cc:348] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } }
I20260926 11:42:55.860734 21414 raft_consensus.cc:729] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddd7c46e0f0f4774be80ac96a09b2f3d, State: Initialized, Role: FOLLOWER
I20260926 11:42:55.861264 21414 consensus_queue.cc:260] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } }
I20260926 11:42:55.862347 21414 ts_tablet_manager.cc:1428] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d: Time spent starting tablet: real 0.005s user 0.006s sys 0.001s
W20260926 11:42:55.919795 21524 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:42:55.841247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.1:43385)
0926 11:42:55.841865 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:55.841926 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:55.843052 (+ 1126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:55.918752 (+ 75700us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":161,"threads_started":1}
W20260926 11:42:55.920320 21522 heartbeater.cc:644] Failed to heartbeat to 127.6.96.62:46729 (0 consecutive failures): Network error: Failed to ping master at 127.6.96.62:46729: Client connection negotiation failed: client connection to 127.6.96.62:46729: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 11:42:55.921038 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:55.920756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.1:52599)
0926 11:42:55.920888 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:55.920968 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260926 11:42:55.921742 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:55.921538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.1:45017)
0926 11:42:55.921631 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:55.921682 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260926 11:42:55.921911 21522 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 11:42:55.953902 21525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:42:55.954140 21525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:42:55.954181 21525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:42:55.958130 21525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:42:55.958184 21525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:42:55.958240 21525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:42:55.963445 21525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:38355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:42979
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=34251
--tserver_master_addrs=127.6.96.62:46729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:42:55.964421 21525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:42:55.965368 21525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:42:55.974498 21533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:55.974514 21525 server_base.cc:1034] running on GCE node
W20260926 11:42:55.974504 21532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:55.974534 21535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:55.975203 21525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:42:55.975819 21525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:42:55.976984 21525 hybrid_clock.cc:648] HybridClock initialized: now 1790422975976944 us; error 54 us; skew 500 ppm
Sep 26 11:42:55 dist-test-slave-xxxx krb5kdc[20214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422975, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:55.979604 21525 init.cc:377] Logged in from keytab as kudu/127.6.96.2@KRBTEST.COM (short username kudu)
I20260926 11:42:55.980458 21525 webserver.cc:460] Webserver started at http://127.6.96.2:34251/ using document root <none> and password file <none>
I20260926 11:42:55.980983 21525 fs_manager.cc:359] Metadata directory not provided
I20260926 11:42:55.981046 21525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:42:55.984771 21525 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260926 11:42:55.986805 21542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:55.988098 21525 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:42:55.988215 21525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "385584ce2fca4815b11220c5443198e7"
format_stamp: "Formatted at 2026-09-26 11:42:53 on dist-test-slave-xxxx"
server_key: "880130e14b3c26f794116ac3e75047c4"
server_key_iv: "01d1b4f3758cab23f679ef77c0ad71ce"
server_key_version: "encryptionkey@0"
I20260926 11:42:55.988755 21525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:42:56.008472 21525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:42:56.011226 21525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:42:56.011404 21525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:42:56.011972 21525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:42:56.012771 21525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:42:56.012817 21525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:56.012871 21525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:42:56.012898 21525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:56.021173 21525 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:42979
I20260926 11:42:56.021193 21655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:42979 every 8 connection(s)
I20260926 11:42:56.022156 21525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:42:56.025998 21658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:56.024375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.2:45677)
0926 11:42:56.024827 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:56.025314 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":122,"threads_started":1}
W20260926 11:42:56.026435 21656 heartbeater.cc:644] Failed to heartbeat to 127.6.96.62:46729 (0 consecutive failures): Network error: Failed to ping master at 127.6.96.62:46729: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
I20260926 11:42:56.027050 21658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:56.026847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.2:42239)
0926 11:42:56.026948 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:56.026994 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 11:42:56.027612 21658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:56.027446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.2:43273)
0926 11:42:56.027531 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:56.027570 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260926 11:42:56.027760 21656 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 11:42:56.031106 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 21525
I20260926 11:42:56.031500 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:35929
--local_ip_for_outbound_sockets=127.6.96.3
--tserver_master_addrs=127.6.96.62:46729
--webserver_port=39795
--webserver_interface=127.6.96.3
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:38355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:42:56.139070 21659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:42:56.139328 21659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:42:56.139370 21659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:42:56.143399 21659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:42:56.143471 21659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:42:56.143530 21659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:42:56.148731 21659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:38355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:35929
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=39795
--tserver_master_addrs=127.6.96.62:46729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:42:56.149780 21659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:42:56.150746 21659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:42:56.160176 21667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:56.160334 21659 server_base.cc:1034] running on GCE node
W20260926 11:42:56.160199 21664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:42:56.160238 21665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:42:56.160830 21659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:42:56.161499 21659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:42:56.162667 21659 hybrid_clock.cc:648] HybridClock initialized: now 1790422976162643 us; error 36 us; skew 500 ppm
Sep 26 11:42:56 dist-test-slave-xxxx krb5kdc[20214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422976, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:42:56.165308 21659 init.cc:377] Logged in from keytab as kudu/127.6.96.3@KRBTEST.COM (short username kudu)
I20260926 11:42:56.166190 21659 webserver.cc:460] Webserver started at http://127.6.96.3:39795/ using document root <none> and password file <none>
I20260926 11:42:56.166734 21659 fs_manager.cc:359] Metadata directory not provided
I20260926 11:42:56.166800 21659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:42:56.170650 21659 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:42:56.172899 21674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:56.174103 21659 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:42:56.174209 21659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "bf1fd9cc5e054d768f61e37e68fbad76"
format_stamp: "Formatted at 2026-09-26 11:42:54 on dist-test-slave-xxxx"
server_key: "ee4cc5aab0a38d55d4aaf75d7b0f68ea"
server_key_iv: "b246bfc2ca05d5726c835fb414f9501f"
server_key_version: "encryptionkey@0"
I20260926 11:42:56.174741 21659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:42:56.197127 21659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:42:56.200155 21659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:42:56.200321 21659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:42:56.200896 21659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:42:56.201964 21681 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260926 11:42:56.205680 21681 ts_tablet_manager.cc:1852] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260926 11:42:56.206060 21681 ts_tablet_manager.cc:1905] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:42:56.209991 21681 ts_tablet_manager.cc:1918] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:42:56.210148 21681 ts_tablet_manager.cc:1939] T 2edfdc182fe64c958a55d8cd737f8e45 P bf1fd9cc5e054d768f61e37e68fbad76: Deleting consensus metadata
I20260926 11:42:56.210558 21659 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260926 11:42:56.210621 21659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.009s user 0.000s sys 0.000s
I20260926 11:42:56.210711 21659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:42:56.210736 21659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:42:56.219177 21659 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:35929
I20260926 11:42:56.219305 21788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:35929 every 8 connection(s)
I20260926 11:42:56.220075 21659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:42:56.223444 21791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:56.222153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.3:42119)
0926 11:42:56.222610 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:56.222724 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":105,"threads_started":1}
W20260926 11:42:56.223917 21789 heartbeater.cc:644] Failed to heartbeat to 127.6.96.62:46729 (0 consecutive failures): Network error: Failed to ping master at 127.6.96.62:46729: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
I20260926 11:42:56.224536 21791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:56.224332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.3:60449)
0926 11:42:56.224439 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:56.224485 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260926 11:42:56.225077 21791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:56.224914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.3:38527)
0926 11:42:56.224992 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:56.225029 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260926 11:42:56.225242 21789 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 11:42:56.227671 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 21659
I20260926 11:42:56.923007 21793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:56.922515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.1:42797)
0926 11:42:56.922778 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:56.922890 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":73,"threads_started":1}
I20260926 11:42:57.028736 21794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:57.028331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.2:39619)
0926 11:42:57.028570 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:57.028629 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":70,"threads_started":1}
I20260926 11:42:57.226202 21795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:57.225766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.3:54435)
0926 11:42:57.226029 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:57.226087 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":75,"threads_started":1}
I20260926 11:42:57.562107 21796 raft_consensus.cc:482] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:42:57.562301 21796 raft_consensus.cc:504] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } }
I20260926 11:42:57.563469 21796 leader_election.cc:288] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf1fd9cc5e054d768f61e37e68fbad76 (127.6.96.3:35929), 385584ce2fca4815b11220c5443198e7 (127.6.96.2:42979)
Sep 26 11:42:57 dist-test-slave-xxxx krb5kdc[20214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422975, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.2@KRBTEST.COM
Sep 26 11:42:57 dist-test-slave-xxxx krb5kdc[20214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422975, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.3@KRBTEST.COM
I20260926 11:42:57.579524 21800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:57.563682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:35929 (local address 127.6.96.1:41773)
0926 11:42:57.564495 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:57.564505 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:57.564589 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:57.566112 (+ 1523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:57.566118 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:57.566707 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:42:57.567518 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:57.567540 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:57.569480 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:57.569483 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:57.571086 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:57.571093 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:57.572116 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:57.572120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:57.572377 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:57.573159 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:42:57.573179 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:42:57.575131 (+ 1952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:42:57.576936 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:57.576939 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:57.576940 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:57.577361 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:57.577724 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:57.577726 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:57.577727 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:57.577867 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:57.578545 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:42:57.578548 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:42:57.578788 (+ 240us) client_negotiation.cc:769] Sending connection context
0926 11:42:57.578992 (+ 204us) client_negotiation.cc:240] Negotiation successful
0926 11:42:57.579337 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"mutex_wait_us":940,"spinlock_wait_cycles":8448,"thread_start_us":59,"threads_started":1}
I20260926 11:42:57.579547 21799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:57.563970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:59963 (local address 127.6.96.2:42979)
0926 11:42:57.564322 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:57.564330 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:57.564360 (+ 30us) server_negotiation.cc:407] Connection header received
0926 11:42:57.565174 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:57.565183 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:57.565637 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:57.566002 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:57.567722 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:57.569316 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:57.571218 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:57.571952 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:57.572127 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:57.575022 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:57.575027 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:57.575040 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:57.575068 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:57.576765 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:57.577475 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:57.577481 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:57.577487 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:57.577612 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:57.577954 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:57.577958 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:57.577960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:57.578424 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:57.578544 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:57.579129 (+ 585us) server_negotiation.cc:299] Negotiation successful
0926 11:42:57.579392 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":133,"threads_started":1}
I20260926 11:42:57.579658 21797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:57.563682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:42979 (local address 127.6.96.1:59963)
0926 11:42:57.564054 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:57.564068 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:42:57.564179 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:42:57.566112 (+ 1933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:42:57.566118 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:42:57.566707 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:42:57.567518 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:57.567540 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:57.569480 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:57.569483 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:57.571083 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:42:57.571093 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:42:57.572088 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:42:57.572092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:42:57.572382 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:57.573244 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:42:57.573257 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:42:57.574748 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:42:57.576881 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:57.576888 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:57.576902 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:57.577354 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:57.577704 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:42:57.577707 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:42:57.577709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:42:57.577857 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:42:57.578536 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:42:57.578542 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:42:57.578792 (+ 250us) client_negotiation.cc:769] Sending connection context
0926 11:42:57.578998 (+ 206us) client_negotiation.cc:240] Negotiation successful
0926 11:42:57.579472 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":501,"spinlock_wait_cycles":56448,"thread_start_us":78,"threads_started":1}
I20260926 11:42:57.580150 21798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:57.563907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:41773 (local address 127.6.96.3:35929)
0926 11:42:57.564220 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0926 11:42:57.564235 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 11:42:57.564595 (+ 360us) server_negotiation.cc:407] Connection header received
0926 11:42:57.565389 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:42:57.565396 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:42:57.565666 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:42:57.566001 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:42:57.567722 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:57.569316 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:57.571207 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:42:57.571952 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:42:57.572154 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:42:57.575323 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:42:57.575328 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:42:57.575340 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:42:57.575368 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:42:57.576830 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:57.577510 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:57.577514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:57.577519 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:57.577631 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:42:57.577964 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:42:57.577966 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:42:57.577968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:42:57.578407 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:42:57.578561 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 11:42:57.579131 (+ 570us) server_negotiation.cc:299] Negotiation successful
0926 11:42:57.579405 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":86,"threads_started":1}
W20260926 11:42:57.581981 21411 leader_election.cc:341] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 385584ce2fca4815b11220c5443198e7 (127.6.96.2:42979): Not found: Tablet not found: 2edfdc182fe64c958a55d8cd737f8e45
W20260926 11:42:57.582113 21410 leader_election.cc:341] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bf1fd9cc5e054d768f61e37e68fbad76 (127.6.96.3:35929): Not found: Tablet not found: 2edfdc182fe64c958a55d8cd737f8e45
I20260926 11:42:57.582207 21410 leader_election.cc:302] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ddd7c46e0f0f4774be80ac96a09b2f3d; no voters: 385584ce2fca4815b11220c5443198e7, bf1fd9cc5e054d768f61e37e68fbad76
I20260926 11:42:57.582484 21796 raft_consensus.cc:2726] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 11:42:57.924120 21800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:57.923764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.1:38405)
0926 11:42:57.923901 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:57.923999 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260926 11:42:58.029874 21801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:58.029468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.2:46075)
0926 11:42:58.029716 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:58.029770 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":70,"threads_started":1}
I20260926 11:42:58.227443 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:58.226931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.3:35805)
0926 11:42:58.227271 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:58.227342 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":76,"threads_started":1}
I20260926 11:42:58.925323 21803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:58.924872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.1:60983)
0926 11:42:58.925159 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:58.925222 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":72,"threads_started":1}
I20260926 11:42:59.030984 21804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:59.030559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.2:46369)
0926 11:42:59.030800 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:59.030890 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":70,"threads_started":1}
I20260926 11:42:59.228628 21805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:59.228148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.3:41519)
0926 11:42:59.228463 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:59.228530 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":73,"threads_started":1}
I20260926 11:42:59.597479 21806 raft_consensus.cc:482] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:42:59.597640 21806 raft_consensus.cc:504] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } }
I20260926 11:42:59.598042 21806 leader_election.cc:288] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf1fd9cc5e054d768f61e37e68fbad76 (127.6.96.3:35929), 385584ce2fca4815b11220c5443198e7 (127.6.96.2:42979)
W20260926 11:42:59.598877 21410 leader_election.cc:341] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bf1fd9cc5e054d768f61e37e68fbad76 (127.6.96.3:35929): Not found: Tablet not found: 2edfdc182fe64c958a55d8cd737f8e45
W20260926 11:42:59.598987 21411 leader_election.cc:341] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 385584ce2fca4815b11220c5443198e7 (127.6.96.2:42979): Not found: Tablet not found: 2edfdc182fe64c958a55d8cd737f8e45
I20260926 11:42:59.599048 21411 leader_election.cc:302] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ddd7c46e0f0f4774be80ac96a09b2f3d; no voters: 385584ce2fca4815b11220c5443198e7, bf1fd9cc5e054d768f61e37e68fbad76
I20260926 11:42:59.599215 21806 raft_consensus.cc:2726] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 11:42:59.926472 21807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:42:59.926033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.1:55037)
0926 11:42:59.926311 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0926 11:42:59.926370 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":92,"threads_started":1}
I20260926 11:43:00.032181 21808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:00.031753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.2:54943)
0926 11:43:00.031999 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:00.032056 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":71,"threads_started":1}
I20260926 11:43:00.229826 21809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:00.229409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.3:44049)
0926 11:43:00.229666 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:00.229725 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":84,"threads_started":1}
I20260926 11:43:00.927798 21810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:00.927354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.1:42643)
0926 11:43:00.927647 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:00.927704 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":90,"threads_started":1}
I20260926 11:43:01.033313 21811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:01.032895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.2:44547)
0926 11:43:01.033150 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:01.033218 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":70,"threads_started":1}
I20260926 11:43:01.231074 21812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:01.230620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.3:42355)
0926 11:43:01.230909 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:01.230973 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":94,"threads_started":1}
I20260926 11:43:01.928880 21813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:01.928472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.1:42173)
0926 11:43:01.928728 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:01.928782 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":92,"threads_started":1}
I20260926 11:43:02.034399 21814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:02.034001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.2:53057)
0926 11:43:02.034249 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:02.034302 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":77,"threads_started":1}
I20260926 11:43:02.232264 21815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:02.231815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.3:38045)
0926 11:43:02.232108 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:02.232170 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":89,"threads_started":1}
I20260926 11:43:02.622301 21816 raft_consensus.cc:482] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:43:02.622430 21816 raft_consensus.cc:504] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } }
I20260926 11:43:02.622845 21816 leader_election.cc:288] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf1fd9cc5e054d768f61e37e68fbad76 (127.6.96.3:35929), 385584ce2fca4815b11220c5443198e7 (127.6.96.2:42979)
W20260926 11:43:02.623534 21410 leader_election.cc:341] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bf1fd9cc5e054d768f61e37e68fbad76 (127.6.96.3:35929): Not found: Tablet not found: 2edfdc182fe64c958a55d8cd737f8e45
W20260926 11:43:02.623642 21411 leader_election.cc:341] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 385584ce2fca4815b11220c5443198e7 (127.6.96.2:42979): Not found: Tablet not found: 2edfdc182fe64c958a55d8cd737f8e45
I20260926 11:43:02.623706 21411 leader_election.cc:302] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ddd7c46e0f0f4774be80ac96a09b2f3d; no voters: 385584ce2fca4815b11220c5443198e7, bf1fd9cc5e054d768f61e37e68fbad76
I20260926 11:43:02.623831 21816 raft_consensus.cc:2726] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 11:43:02.929991 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:02.929599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.1:59203)
0926 11:43:02.929837 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:02.929892 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":74,"threads_started":1}
I20260926 11:43:03.035652 21818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:03.035165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.2:48001)
0926 11:43:03.035489 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:03.035555 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":72,"threads_started":1}
I20260926 11:43:03.233448 21819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:03.232978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.3:43127)
0926 11:43:03.233297 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:03.233353 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":107,"threads_started":1}
I20260926 11:43:03.931160 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:03.930674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.1:51665)
0926 11:43:03.930996 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:03.931051 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":97,"threads_started":1}
I20260926 11:43:04.036851 21821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:04.036423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.2:60295)
0926 11:43:04.036693 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:04.036756 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":89,"threads_started":1}
I20260926 11:43:04.234774 21822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:04.234289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.3:44595)
0926 11:43:04.234607 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:04.234665 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":88,"threads_started":1}
I20260926 11:43:04.837920 21823 raft_consensus.cc:482] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:43:04.838076 21823 raft_consensus.cc:504] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf1fd9cc5e054d768f61e37e68fbad76" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35929 } } peers { permanent_uuid: "385584ce2fca4815b11220c5443198e7" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 42979 } } peers { permanent_uuid: "ddd7c46e0f0f4774be80ac96a09b2f3d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 40911 } }
I20260926 11:43:04.838490 21823 leader_election.cc:288] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf1fd9cc5e054d768f61e37e68fbad76 (127.6.96.3:35929), 385584ce2fca4815b11220c5443198e7 (127.6.96.2:42979)
W20260926 11:43:04.839221 21410 leader_election.cc:341] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bf1fd9cc5e054d768f61e37e68fbad76 (127.6.96.3:35929): Not found: Tablet not found: 2edfdc182fe64c958a55d8cd737f8e45
W20260926 11:43:04.839320 21411 leader_election.cc:341] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 385584ce2fca4815b11220c5443198e7 (127.6.96.2:42979): Not found: Tablet not found: 2edfdc182fe64c958a55d8cd737f8e45
I20260926 11:43:04.839408 21411 leader_election.cc:302] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ddd7c46e0f0f4774be80ac96a09b2f3d; no voters: 385584ce2fca4815b11220c5443198e7, bf1fd9cc5e054d768f61e37e68fbad76
I20260926 11:43:04.839560 21823 raft_consensus.cc:2726] T 2edfdc182fe64c958a55d8cd737f8e45 P ddd7c46e0f0f4774be80ac96a09b2f3d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 11:43:04.932262 21824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:04.931821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.1:45261)
0926 11:43:04.932095 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:04.932165 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":112,"threads_started":1}
I20260926 11:43:05.038055 21825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:05.037619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.2:41575)
0926 11:43:05.037883 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:05.037958 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":75,"threads_started":1}
I20260926 11:43:05.236001 21826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:05.235569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.3:46657)
0926 11:43:05.235843 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:05.235899 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":84,"threads_started":1}
I20260926 11:43:05.933600 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:05.933151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.1:43993)
0926 11:43:05.933438 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:05.933502 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":77,"threads_started":1}
I20260926 11:43:06.039237 21828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:06.038813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.2:40117)
0926 11:43:06.039069 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:06.039123 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":74,"threads_started":1}
I20260926 11:43:06.237286 21829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:06.236815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.3:58885)
0926 11:43:06.237120 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:06.237180 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":121,"threads_started":1}
I20260926 11:43:06.934938 21830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:06.934393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.1:45497)
0926 11:43:06.934694 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:06.934783 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":77,"threads_started":1}
I20260926 11:43:07.040586 21831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:07.040113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.2:46645)
0926 11:43:07.040420 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:07.040488 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":81,"threads_started":1}
I20260926 11:43:07.238687 21832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:07.238201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.3:46963)
0926 11:43:07.238480 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:07.238556 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":89,"threads_started":1}
I20260926 11:43:07.936200 21833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:07.935761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.1:33199)
0926 11:43:07.936039 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:07.936099 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":80,"threads_started":1}
I20260926 11:43:08.041841 21834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:08.041395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.2:43193)
0926 11:43:08.041669 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:08.041736 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":71,"threads_started":1}
I20260926 11:43:08.239903 21835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:08.239447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.3:49221)
0926 11:43:08.239751 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:08.239812 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":90,"threads_started":1}
I20260926 11:43:08.937520 21836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:08.937054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.1:50613)
0926 11:43:08.937356 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:08.937422 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":94,"threads_started":1}
I20260926 11:43:09.043100 21837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:09.042626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.2:58233)
0926 11:43:09.042926 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:09.042994 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":96,"threads_started":1}
I20260926 11:43:09.241204 21838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:09.240715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.3:43573)
0926 11:43:09.241043 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:09.241106 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":107,"threads_started":1}
I20260926 11:43:09.938781 21839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:09.938324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.1:41253)
0926 11:43:09.938615 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:09.938678 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":80,"threads_started":1}
I20260926 11:43:10.044425 21840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:10.043948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.2:55539)
0926 11:43:10.044257 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:10.044324 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":81,"threads_started":1}
I20260926 11:43:10.242444 21841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:10.241978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.3:36233)
0926 11:43:10.242267 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:10.242346 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":77,"threads_started":1}
I20260926 11:43:10.940464 21842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:10.939972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.1:41977)
0926 11:43:10.940271 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:10.940364 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":89,"threads_started":1}
I20260926 11:43:11.045926 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:11.045460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.2:57009)
0926 11:43:11.045748 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:11.045826 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":75,"threads_started":1}
I20260926 11:43:11.240283 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 21358
I20260926 11:43:11.243956 21844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:11.243485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:46729 (local address 127.6.96.3:44011)
0926 11:43:11.243781 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:11.243853 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:46729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":83,"threads_started":1}
I20260926 11:43:11.246166 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 21525
I20260926 11:43:11.251174 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 21659
2026-09-26T11:43:11Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19483 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:43:11 dist-test-slave-xxxx krb5kdc[21847](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:43:11 dist-test-slave-xxxx krb5kdc[21847](info): set up 2 sockets
Sep 26 11:43:11 dist-test-slave-xxxx krb5kdc[21847](info): commencing operation
krb5kdc: starting...
W20260926 11:43:12.690734 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:43:12 dist-test-slave-xxxx krb5kdc[21847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:43:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:43:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:43:12.818699 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:45691
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:45271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:45691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:43:12.931944 21860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:43:12.932226 21860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:43:12.932286 21860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:43:12.936381 21860 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 11:43:12.936444 21860 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 11:43:12.936465 21860 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 11:43:12.936481 21860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:43:12.936507 21860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:43:12.936540 21860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:43:12.936571 21860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:43:12.936599 21860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:43:12.942260 21860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:45271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:45691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:45691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:43:12.943583 21860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:43:12.944659 21860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:43:12.953310 21866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:43:12.953336 21865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:43:12.953291 21868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:43:12.953622 21860 server_base.cc:1034] running on GCE node
I20260926 11:43:12.954248 21860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:43:12.955487 21860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:43:12.956678 21860 hybrid_clock.cc:648] HybridClock initialized: now 1790422992956646 us; error 42 us; skew 500 ppm
Sep 26 11:43:12 dist-test-slave-xxxx krb5kdc[21847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422992, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:43:12.966147 21860 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:43:12.967229 21860 webserver.cc:460] Webserver started at http://127.6.96.62:36629/ using document root <none> and password file <none>
I20260926 11:43:12.967798 21860 fs_manager.cc:359] Metadata directory not provided
I20260926 11:43:12.967868 21860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:43:12.968075 21860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:43:12.970219 21860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "6395ba7907354546ae2ae6772a73a5d9"
format_stamp: "Formatted at 2026-09-26 11:43:12 on dist-test-slave-xxxx"
server_key: "481ac7e305a6400a47feaff7f8c77813"
server_key_iv: "489fec93914a580bc666f848124e9627"
server_key_version: "encryptionkey@0"
I20260926 11:43:12.970815 21860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "6395ba7907354546ae2ae6772a73a5d9"
format_stamp: "Formatted at 2026-09-26 11:43:12 on dist-test-slave-xxxx"
server_key: "481ac7e305a6400a47feaff7f8c77813"
server_key_iv: "489fec93914a580bc666f848124e9627"
server_key_version: "encryptionkey@0"
I20260926 11:43:12.974748 21860 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 11:43:12.977185 21875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:12.978477 21860 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 11:43:12.978598 21860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "6395ba7907354546ae2ae6772a73a5d9"
format_stamp: "Formatted at 2026-09-26 11:43:12 on dist-test-slave-xxxx"
server_key: "481ac7e305a6400a47feaff7f8c77813"
server_key_iv: "489fec93914a580bc666f848124e9627"
server_key_version: "encryptionkey@0"
I20260926 11:43:12.978700 21860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:43:12.991102 21860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:43:12.994313 21860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:43:12.994534 21860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:43:13.001587 21860 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:45691
I20260926 11:43:13.001843 21927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:45691 every 8 connection(s)
I20260926 11:43:13.002616 21860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:43:13.005101 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 21860
I20260926 11:43:13.005237 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:43:13.005551 6528 external_mini_cluster.cc:1428] Setting key 6230edc92f8c6a206dd485ddd2ed5239
I20260926 11:43:13.005668 21928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:43:13.014082 21928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9: Bootstrap starting.
Sep 26 11:43:13 dist-test-slave-xxxx krb5kdc[21847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:43:13.017129 21928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9: Neither blocks nor log segments found. Creating new log.
I20260926 11:43:13.017980 21928 log.cc:824] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9: Log is configured to *not* fsync() on all Append() calls
I20260926 11:43:13.020351 21928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9: No bootstrap required, opened a new log
I20260926 11:43:13.023561 21931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:13.007061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46398 (local address 127.6.96.62:45691)
0926 11:43:13.007592 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:13.007608 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:13.007655 (+ 47us) server_negotiation.cc:407] Connection header received
0926 11:43:13.008838 (+ 1183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:13.008867 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:13.009369 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:13.009781 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:43:13.010848 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.012437 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.013515 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.014148 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.014482 (+ 334us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:13.017476 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:43:13.017483 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:43:13.017505 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:43:13.017546 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:43:13.020022 (+ 2476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:13.020540 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:13.020547 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:13.020554 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:13.020675 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:13.020961 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:13.020965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:13.020967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:13.021489 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:43:13.021695 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0926 11:43:13.022031 (+ 336us) server_negotiation.cc:299] Negotiation successful
0926 11:43:13.022421 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":150,"threads_started":1}
I20260926 11:43:13.025146 21928 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6395ba7907354546ae2ae6772a73a5d9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45691 } }
I20260926 11:43:13.025460 21928 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:43:13.025522 21928 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6395ba7907354546ae2ae6772a73a5d9, State: Initialized, Role: FOLLOWER
I20260926 11:43:13.026090 21928 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6395ba7907354546ae2ae6772a73a5d9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45691 } }
I20260926 11:43:13.026237 21928 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:43:13.026296 21928 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:43:13.026386 21928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:43:13.027382 21928 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6395ba7907354546ae2ae6772a73a5d9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45691 } }
I20260926 11:43:13.027710 21928 leader_election.cc:302] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6395ba7907354546ae2ae6772a73a5d9; no voters:
I20260926 11:43:13.028023 21928 leader_election.cc:288] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:43:13.028115 21933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:43:13.028316 21933 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 1 LEADER]: Becoming Leader. State: Replica: 6395ba7907354546ae2ae6772a73a5d9, State: Running, Role: LEADER
I20260926 11:43:13.028661 21933 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6395ba7907354546ae2ae6772a73a5d9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45691 } }
I20260926 11:43:13.029273 21928 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:43:13.030212 21935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6395ba7907354546ae2ae6772a73a5d9. Latest consensus state: current_term: 1 leader_uuid: "6395ba7907354546ae2ae6772a73a5d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6395ba7907354546ae2ae6772a73a5d9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45691 } } }
I20260926 11:43:13.030350 21935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [sys.catalog]: This master's current role is: LEADER
I20260926 11:43:13.030747 21942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:43:13.030781 21934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6395ba7907354546ae2ae6772a73a5d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6395ba7907354546ae2ae6772a73a5d9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45691 } } }
I20260926 11:43:13.031028 21934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [sys.catalog]: This master's current role is: LEADER
I20260926 11:43:13.033805 21942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:43:13.039331 21942 catalog_manager.cc:1347] Generated new cluster ID: 8eae91f9b435482a8e9d2f9d9dde6031
I20260926 11:43:13.039412 21942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:43:13.054562 21942 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:43:13.055770 21942 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:43:13.063877 21942 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9: Generated new TSK 0
I20260926 11:43:13.064597 21942 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.96.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
I20260926 11:43:13.096164 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:45691
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:45271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:43:13.207736 21954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:43:13.208060 21954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:43:13.208125 21954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:43:13.212291 21954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:43:13.212369 21954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:43:13.212452 21954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:43:13.217919 21954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:45271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:45691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:43:13.219098 21954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:43:13.220166 21954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 11:43:13.229851 21954 server_base.cc:1034] running on GCE node
W20260926 11:43:13.229794 21960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:43:13.229789 21959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:43:13.229794 21962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:43:13.230494 21954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:43:13.231271 21954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:43:13.232427 21954 hybrid_clock.cc:648] HybridClock initialized: now 1790422993232394 us; error 46 us; skew 500 ppm
Sep 26 11:43:13 dist-test-slave-xxxx krb5kdc[21847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422993, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:43:13.235132 21954 init.cc:377] Logged in from keytab as kudu/127.6.96.1@KRBTEST.COM (short username kudu)
I20260926 11:43:13.236001 21954 webserver.cc:460] Webserver started at http://127.6.96.1:46711/ using document root <none> and password file <none>
I20260926 11:43:13.236523 21954 fs_manager.cc:359] Metadata directory not provided
I20260926 11:43:13.236589 21954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:43:13.236778 21954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:43:13.238778 21954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "38b24333651147528537f839a2085d84"
format_stamp: "Formatted at 2026-09-26 11:43:13 on dist-test-slave-xxxx"
server_key: "324a2e58deb6fe391d9743567b8da01a"
server_key_iv: "5ed8a67423014a80c6574189d1c82c49"
server_key_version: "encryptionkey@0"
I20260926 11:43:13.239364 21954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "38b24333651147528537f839a2085d84"
format_stamp: "Formatted at 2026-09-26 11:43:13 on dist-test-slave-xxxx"
server_key: "324a2e58deb6fe391d9743567b8da01a"
server_key_iv: "5ed8a67423014a80c6574189d1c82c49"
server_key_version: "encryptionkey@0"
I20260926 11:43:13.243181 21954 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 11:43:13.245705 21969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:13.247092 21954 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:43:13.247207 21954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "38b24333651147528537f839a2085d84"
format_stamp: "Formatted at 2026-09-26 11:43:13 on dist-test-slave-xxxx"
server_key: "324a2e58deb6fe391d9743567b8da01a"
server_key_iv: "5ed8a67423014a80c6574189d1c82c49"
server_key_version: "encryptionkey@0"
I20260926 11:43:13.247332 21954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:43:13.263921 21954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:43:13.267171 21954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:43:13.267355 21954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:43:13.267936 21954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:43:13.268800 21954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:43:13.268846 21954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:13.268905 21954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:43:13.268931 21954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:13.277285 21954 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:44571
I20260926 11:43:13.277307 22082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:44571 every 8 connection(s)
I20260926 11:43:13.278184 21954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:43:13.282971 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 21954
I20260926 11:43:13.283073 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:43:13.283340 6528 external_mini_cluster.cc:1428] Setting key 18600472f49cd41337bd697c51a78a30
Sep 26 11:43:13 dist-test-slave-xxxx krb5kdc[21847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422993, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.96.2@KRBTEST.COM; defaulting to no policy
I20260926 11:43:13.292726 21931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:13.280023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:40953 (local address 127.6.96.62:45691)
0926 11:43:13.280182 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:13.280185 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:13.281239 (+ 1054us) server_negotiation.cc:407] Connection header received
0926 11:43:13.282163 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:13.282167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:13.282214 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:13.282351 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:43:13.283714 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.284336 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.285560 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.285882 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.285989 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:13.289284 (+ 3295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:43:13.289288 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:43:13.289290 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:43:13.289317 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:43:13.290612 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:13.291156 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:13.291159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:13.291161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:13.291239 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:13.291564 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:13.291566 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:13.291567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:13.291762 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:43:13.291852 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 11:43:13.292443 (+ 591us) server_negotiation.cc:299] Negotiation successful
0926 11:43:13.292573 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 11:43:13.293563 22085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:13.280384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.1:40953)
0926 11:43:13.281089 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:13.281132 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 11:43:13.281957 (+ 825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:43:13.282501 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:43:13.282510 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:43:13.282981 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:43:13.283569 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:13.283585 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.284468 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.284471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:13.285440 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:13.285450 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.286055 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.286060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:13.286256 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:13.286868 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:43:13.286895 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:43:13.288583 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:43:13.290730 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:43:13.290737 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:43:13.290755 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:43:13.291061 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:43:13.291324 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:43:13.291327 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:43:13.291329 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:43:13.291468 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:43:13.291864 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:43:13.291879 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:43:13.292168 (+ 289us) client_negotiation.cc:769] Sending connection context
0926 11:43:13.292339 (+ 171us) client_negotiation.cc:240] Negotiation successful
0926 11:43:13.292668 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":115,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.2@KRBTEST.COM" created.
I20260926 11:43:13.294885 22083 heartbeater.cc:347] Connected to a master server at 127.6.96.62:45691
I20260926 11:43:13.295128 22083 heartbeater.cc:464] Registering TS with master...
I20260926 11:43:13.295632 22083 heartbeater.cc:511] Master 127.6.96.62:45691 requested a full tablet report, sending...
I20260926 11:43:13.297260 21889 ts_manager.cc:194] Registered new tserver with Master: 38b24333651147528537f839a2085d84 (127.6.96.1:44571)
I20260926 11:43:13.298373 21889 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.1@KRBTEST.COM'} at 127.6.96.1:40953
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
I20260926 11:43:13.310384 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:45691
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:45271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:43:13.420123 22088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:43:13.420401 22088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:43:13.420461 22088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:43:13.424877 22088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:43:13.424949 22088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:43:13.425032 22088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:43:13.430621 22088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:45271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:45691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:43:13.431806 22088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:43:13.432806 22088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:43:13.442855 22093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:43:13.442855 22094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:43:13.443027 22088 server_base.cc:1034] running on GCE node
W20260926 11:43:13.443186 22096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:43:13.443585 22088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:43:13.444269 22088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:43:13.445433 22088 hybrid_clock.cc:648] HybridClock initialized: now 1790422993445414 us; error 36 us; skew 500 ppm
Sep 26 11:43:13 dist-test-slave-xxxx krb5kdc[21847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422993, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:43:13.448360 22088 init.cc:377] Logged in from keytab as kudu/127.6.96.2@KRBTEST.COM (short username kudu)
I20260926 11:43:13.449375 22088 webserver.cc:460] Webserver started at http://127.6.96.2:43043/ using document root <none> and password file <none>
I20260926 11:43:13.449929 22088 fs_manager.cc:359] Metadata directory not provided
I20260926 11:43:13.449990 22088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:43:13.450192 22088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:43:13.452345 22088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "520628b8ec3e40a6837bd876ebc09433"
format_stamp: "Formatted at 2026-09-26 11:43:13 on dist-test-slave-xxxx"
server_key: "ba1e388515aa42c7838d86eb783e303e"
server_key_iv: "451a289daf1c4f746f90ce8b7d821cdb"
server_key_version: "encryptionkey@0"
I20260926 11:43:13.452888 22088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "520628b8ec3e40a6837bd876ebc09433"
format_stamp: "Formatted at 2026-09-26 11:43:13 on dist-test-slave-xxxx"
server_key: "ba1e388515aa42c7838d86eb783e303e"
server_key_iv: "451a289daf1c4f746f90ce8b7d821cdb"
server_key_version: "encryptionkey@0"
I20260926 11:43:13.456704 22088 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260926 11:43:13.459107 22103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:13.460314 22088 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 11:43:13.460428 22088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "520628b8ec3e40a6837bd876ebc09433"
format_stamp: "Formatted at 2026-09-26 11:43:13 on dist-test-slave-xxxx"
server_key: "ba1e388515aa42c7838d86eb783e303e"
server_key_iv: "451a289daf1c4f746f90ce8b7d821cdb"
server_key_version: "encryptionkey@0"
I20260926 11:43:13.460525 22088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:43:13.476936 22088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:43:13.480099 22088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:43:13.480275 22088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:43:13.480868 22088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:43:13.481655 22088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:43:13.481700 22088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:13.481751 22088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:43:13.481786 22088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:13.490049 22088 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:44021
I20260926 11:43:13.490077 22216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:44021 every 8 connection(s)
I20260926 11:43:13.490947 22088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:43:13.496730 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 22088
I20260926 11:43:13.496848 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:43:13.497104 6528 external_mini_cluster.cc:1428] Setting key 903412af3f8068eda9a7acc152141a14
Sep 26 11:43:13 dist-test-slave-xxxx krb5kdc[21847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422993, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:43:13.504493 21931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:13.492674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:42911 (local address 127.6.96.62:45691)
0926 11:43:13.492892 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:13.492897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:13.493886 (+ 989us) server_negotiation.cc:407] Connection header received
0926 11:43:13.494727 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:13.494730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:13.494787 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:13.494922 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:43:13.496053 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.496588 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.497968 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.498199 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.498323 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:13.500919 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:43:13.500923 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:43:13.500925 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:43:13.500950 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:43:13.502229 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:13.502872 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:13.502876 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:13.502878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:13.502962 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:13.503350 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:13.503353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:13.503354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:13.503533 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:43:13.503604 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 11:43:13.504241 (+ 637us) server_negotiation.cc:299] Negotiation successful
0926 11:43:13.504367 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 11:43:13.505645 22219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:13.493084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.2:42911)
0926 11:43:13.493729 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:13.493770 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 11:43:13.494532 (+ 762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:43:13.495083 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:43:13.495089 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:43:13.495441 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:43:13.495928 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:13.495941 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.496729 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.496732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:13.497830 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:13.497842 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.498391 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.498395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:13.498587 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:13.499182 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:43:13.499199 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:43:13.500738 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:43:13.502343 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:43:13.502349 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:43:13.502366 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:43:13.502712 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:43:13.503058 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:43:13.503061 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:43:13.503062 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:43:13.503266 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:43:13.503626 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:43:13.503634 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:43:13.503973 (+ 339us) client_negotiation.cc:769] Sending connection context
0926 11:43:13.504190 (+ 217us) client_negotiation.cc:240] Negotiation successful
0926 11:43:13.504566 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for kudu/127.6.96.3@KRBTEST.COM; defaulting to no policy
I20260926 11:43:13.506855 22217 heartbeater.cc:347] Connected to a master server at 127.6.96.62:45691
I20260926 11:43:13.507090 22217 heartbeater.cc:464] Registering TS with master...
I20260926 11:43:13.507560 22217 heartbeater.cc:511] Master 127.6.96.62:45691 requested a full tablet report, sending...
I20260926 11:43:13.508622 21889 ts_manager.cc:194] Registered new tserver with Master: 520628b8ec3e40a6837bd876ebc09433 (127.6.96.2:44021)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.3@KRBTEST.COM" created.
I20260926 11:43:13.509301 21889 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.2@KRBTEST.COM'} at 127.6.96.2:42911
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
I20260926 11:43:13.523991 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:45691
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:45271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:43:13.632846 22222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:43:13.633129 22222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:43:13.633173 22222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:43:13.637239 22222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:43:13.637302 22222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:43:13.637362 22222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:43:13.642697 22222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:45271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:45691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:43:13.643849 22222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:43:13.644836 22222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 11:43:13.654608 22222 server_base.cc:1034] running on GCE node
W20260926 11:43:13.654585 22230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:43:13.654807 22228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:43:13.654579 22227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:43:13.655280 22222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:43:13.655997 22222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:43:13.657174 22222 hybrid_clock.cc:648] HybridClock initialized: now 1790422993657140 us; error 53 us; skew 500 ppm
Sep 26 11:43:13 dist-test-slave-xxxx krb5kdc[21847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422993, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:43:13.660087 22222 init.cc:377] Logged in from keytab as kudu/127.6.96.3@KRBTEST.COM (short username kudu)
I20260926 11:43:13.661029 22222 webserver.cc:460] Webserver started at http://127.6.96.3:35027/ using document root <none> and password file <none>
I20260926 11:43:13.661589 22222 fs_manager.cc:359] Metadata directory not provided
I20260926 11:43:13.661674 22222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:43:13.661890 22222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:43:13.664111 22222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "6afe305187804e80882f2700bdd14972"
format_stamp: "Formatted at 2026-09-26 11:43:13 on dist-test-slave-xxxx"
server_key: "29f55bcb297d9fe96efe4145e68c4fa0"
server_key_iv: "4885cf12e371264cab2599288556efd1"
server_key_version: "encryptionkey@0"
I20260926 11:43:13.664717 22222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "6afe305187804e80882f2700bdd14972"
format_stamp: "Formatted at 2026-09-26 11:43:13 on dist-test-slave-xxxx"
server_key: "29f55bcb297d9fe96efe4145e68c4fa0"
server_key_iv: "4885cf12e371264cab2599288556efd1"
server_key_version: "encryptionkey@0"
I20260926 11:43:13.668684 22222 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 11:43:13.671358 22237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:13.672803 22222 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:43:13.672933 22222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "6afe305187804e80882f2700bdd14972"
format_stamp: "Formatted at 2026-09-26 11:43:13 on dist-test-slave-xxxx"
server_key: "29f55bcb297d9fe96efe4145e68c4fa0"
server_key_iv: "4885cf12e371264cab2599288556efd1"
server_key_version: "encryptionkey@0"
I20260926 11:43:13.673043 22222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:43:13.691929 22222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:43:13.695233 22222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:43:13.695431 22222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:43:13.696075 22222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:43:13.696904 22222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:43:13.696951 22222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:13.697001 22222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:43:13.697031 22222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:13.705478 22222 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:34785
I20260926 11:43:13.705518 22350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:34785 every 8 connection(s)
I20260926 11:43:13.706362 22222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:43:13.710175 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 22222
I20260926 11:43:13.710321 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:43:13.710626 6528 external_mini_cluster.cc:1428] Setting key 03df71e10357b5c344d46b6fcca6658a
Sep 26 11:43:13 dist-test-slave-xxxx krb5kdc[21847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790422993, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:43:13.719460 21931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:13.708263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:45817 (local address 127.6.96.62:45691)
0926 11:43:13.708418 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:13.708421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:13.709481 (+ 1060us) server_negotiation.cc:407] Connection header received
0926 11:43:13.710432 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:13.710435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:13.710494 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:13.710595 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:43:13.711913 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.712437 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.713539 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.713756 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.713849 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:13.716218 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:43:13.716221 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:43:13.716224 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:43:13.716248 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:43:13.717522 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:13.717999 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:13.718003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:13.718005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:13.718082 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:13.718404 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:13.718407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:13.718409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:13.718585 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:43:13.718671 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 11:43:13.719182 (+ 511us) server_negotiation.cc:299] Negotiation successful
0926 11:43:13.719301 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 11:43:13.720239 22353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:13.708645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.3:45817)
0926 11:43:13.709307 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:13.709372 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0926 11:43:13.710195 (+ 823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:43:13.710756 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:43:13.710762 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:43:13.711180 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:43:13.711785 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:13.711800 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.712551 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.712553 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:13.713429 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:13.713438 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.713890 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.713894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:13.714059 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:13.714655 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:43:13.714671 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:43:13.716069 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:43:13.717622 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:43:13.717628 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:43:13.717654 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:43:13.717908 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:43:13.718162 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:43:13.718165 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:43:13.718167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:43:13.718313 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:43:13.718684 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:43:13.718691 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:43:13.718965 (+ 274us) client_negotiation.cc:769] Sending connection context
0926 11:43:13.719130 (+ 165us) client_negotiation.cc:240] Negotiation successful
0926 11:43:13.719466 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":115,"threads_started":1}
I20260926 11:43:13.721339 22351 heartbeater.cc:347] Connected to a master server at 127.6.96.62:45691
I20260926 11:43:13.721568 22351 heartbeater.cc:464] Registering TS with master...
I20260926 11:43:13.722065 22351 heartbeater.cc:511] Master 127.6.96.62:45691 requested a full tablet report, sending...
I20260926 11:43:13.723086 21889 ts_manager.cc:194] Registered new tserver with Master: 6afe305187804e80882f2700bdd14972 (127.6.96.3:34785)
I20260926 11:43:13.723711 21889 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.3@KRBTEST.COM'} at 127.6.96.3:45817
I20260926 11:43:13.724488 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 11:43:13.733248 21931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:13.726411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46404 (local address 127.6.96.62:45691)
0926 11:43:13.726542 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:13.726546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:13.726761 (+ 215us) server_negotiation.cc:407] Connection header received
0926 11:43:13.726872 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:13.726875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:13.726922 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:13.726983 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:43:13.727628 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.728158 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.729019 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.729230 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.729326 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:13.730591 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:43:13.730594 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:43:13.730596 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:43:13.730617 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:43:13.731903 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:13.732233 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:13.732235 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:13.732236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:13.732315 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:13.732532 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:13.732535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:13.732537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:13.732778 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:43:13.732883 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 11:43:13.733031 (+ 148us) server_negotiation.cc:299] Negotiation successful
0926 11:43:13.733127 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 11:43:13.735780 21889 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 11:43:13.738654 21889 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:43:13.753332 22362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:13.746844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:44021 (local address 127.0.0.1:36100)
0926 11:43:13.747144 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:13.747189 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 11:43:13.747305 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:43:13.748252 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:43:13.748258 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:43:13.748282 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:43:13.748436 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:13.748443 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.750050 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.750053 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:13.751456 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:13.751467 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.752759 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.752762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:13.752892 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:13.752956 (+ 64us) client_negotiation.cc:769] Sending connection context
0926 11:43:13.753059 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 11:43:13.753181 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":3,"thread_start_us":64,"threads_started":1}
I20260926 11:43:13.753356 22363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:13.746982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36100 (local address 127.6.96.2:44021)
0926 11:43:13.747749 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:13.747762 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:13.747788 (+ 26us) server_negotiation.cc:407] Connection header received
0926 11:43:13.747868 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:13.747874 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:13.748075 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:13.748598 (+ 523us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:43:13.748783 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.749899 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.751812 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.752642 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.752749 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:13.752907 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 11:43:13.753125 (+ 218us) server_negotiation.cc:299] Negotiation successful
0926 11:43:13.753223 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":99,"threads_started":1}
I20260926 11:43:13.754014 22365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:13.747529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:34785 (local address 127.0.0.1:41446)
0926 11:43:13.747746 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:13.747759 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:43:13.747877 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:43:13.749145 (+ 1268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:43:13.749148 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:43:13.749222 (+ 74us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:43:13.749399 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:13.749406 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.751567 (+ 2161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.751570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:13.752728 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:13.752737 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.753725 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.753728 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:13.753806 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:13.753823 (+ 17us) client_negotiation.cc:769] Sending connection context
0926 11:43:13.753871 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 11:43:13.753916 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":24,"thread_start_us":89,"threads_started":1}
I20260926 11:43:13.754148 22364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:13.747074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41446 (local address 127.6.96.3:34785)
0926 11:43:13.748685 (+ 1611us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:13.748697 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:13.748722 (+ 25us) server_negotiation.cc:407] Connection header received
0926 11:43:13.748807 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:13.748812 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:13.749022 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:13.749174 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:43:13.749590 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.751464 (+ 1874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.752863 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.753612 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.753721 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:13.753872 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 11:43:13.753922 (+ 50us) server_negotiation.cc:299] Negotiation successful
0926 11:43:13.754006 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1491,"thread_start_us":101,"threads_started":1}
I20260926 11:43:13.755502 22367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:13.747809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:44571 (local address 127.0.0.1:52624)
0926 11:43:13.748362 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:13.748375 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:43:13.748455 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:43:13.750489 (+ 2034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:43:13.750493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:43:13.750509 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:43:13.750607 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:13.750612 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.751937 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.751940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:13.752961 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:13.752968 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.754975 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.754979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:13.755126 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:13.755151 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 11:43:13.755244 (+ 93us) client_negotiation.cc:240] Negotiation successful
0926 11:43:13.755349 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":22,"thread_start_us":357,"threads_started":1}
I20260926 11:43:13.755992 22366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:13.748000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52624 (local address 127.6.96.1:44571)
0926 11:43:13.749887 (+ 1887us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:13.749903 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:13.749925 (+ 22us) server_negotiation.cc:407] Connection header received
0926 11:43:13.749999 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:13.750003 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:13.750183 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:13.750339 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:43:13.750724 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.751802 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.753074 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.754819 (+ 1745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.754937 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:13.755073 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 11:43:13.755804 (+ 731us) server_negotiation.cc:299] Negotiation successful
0926 11:43:13.755886 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1784,"thread_start_us":117,"threads_started":1}
I20260926 11:43:13.756764 22151 tablet_service.cc:1467] Processing CreateTablet for tablet 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 (DEFAULT_TABLE table=test-table [id=b328afa742ef454fbce959199899e9d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:43:13.757015 22285 tablet_service.cc:1467] Processing CreateTablet for tablet 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 (DEFAULT_TABLE table=test-table [id=b328afa742ef454fbce959199899e9d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:43:13.757927 22285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e7d1f44c3b04f6a86c88cb3e6a9ffc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:43:13.757925 22151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e7d1f44c3b04f6a86c88cb3e6a9ffc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:43:13.758245 22017 tablet_service.cc:1467] Processing CreateTablet for tablet 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 (DEFAULT_TABLE table=test-table [id=b328afa742ef454fbce959199899e9d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:43:13.759152 22017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e7d1f44c3b04f6a86c88cb3e6a9ffc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:43:13.764931 22368 tablet_bootstrap.cc:492] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 520628b8ec3e40a6837bd876ebc09433: Bootstrap starting.
I20260926 11:43:13.765686 22369 tablet_bootstrap.cc:492] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84: Bootstrap starting.
I20260926 11:43:13.766112 22370 tablet_bootstrap.cc:492] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972: Bootstrap starting.
I20260926 11:43:13.767755 22370 tablet_bootstrap.cc:654] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972: Neither blocks nor log segments found. Creating new log.
I20260926 11:43:13.767805 22369 tablet_bootstrap.cc:654] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84: Neither blocks nor log segments found. Creating new log.
I20260926 11:43:13.767791 22368 tablet_bootstrap.cc:654] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 520628b8ec3e40a6837bd876ebc09433: Neither blocks nor log segments found. Creating new log.
I20260926 11:43:13.768687 22370 log.cc:824] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972: Log is configured to *not* fsync() on all Append() calls
I20260926 11:43:13.768690 22368 log.cc:824] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 520628b8ec3e40a6837bd876ebc09433: Log is configured to *not* fsync() on all Append() calls
I20260926 11:43:13.768925 22369 log.cc:824] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84: Log is configured to *not* fsync() on all Append() calls
I20260926 11:43:13.770691 22369 tablet_bootstrap.cc:492] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84: No bootstrap required, opened a new log
I20260926 11:43:13.770694 22370 tablet_bootstrap.cc:492] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972: No bootstrap required, opened a new log
I20260926 11:43:13.770862 22368 tablet_bootstrap.cc:492] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 520628b8ec3e40a6837bd876ebc09433: No bootstrap required, opened a new log
I20260926 11:43:13.770903 22370 ts_tablet_manager.cc:1397] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260926 11:43:13.770903 22369 ts_tablet_manager.cc:1397] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260926 11:43:13.771037 22368 ts_tablet_manager.cc:1397] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 520628b8ec3e40a6837bd876ebc09433: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 11:43:13.773774 22368 raft_consensus.cc:348] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 520628b8ec3e40a6837bd876ebc09433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:43:13.774014 22368 raft_consensus.cc:374] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 520628b8ec3e40a6837bd876ebc09433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:43:13.774065 22368 raft_consensus.cc:729] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 520628b8ec3e40a6837bd876ebc09433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 520628b8ec3e40a6837bd876ebc09433, State: Initialized, Role: FOLLOWER
I20260926 11:43:13.774492 22368 consensus_queue.cc:260] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 520628b8ec3e40a6837bd876ebc09433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:43:13.774609 22369 raft_consensus.cc:348] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:43:13.774609 22370 raft_consensus.cc:348] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:43:13.774878 22370 raft_consensus.cc:374] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:43:13.774876 22369 raft_consensus.cc:374] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:43:13.774935 22370 raft_consensus.cc:729] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6afe305187804e80882f2700bdd14972, State: Initialized, Role: FOLLOWER
I20260926 11:43:13.774935 22369 raft_consensus.cc:729] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b24333651147528537f839a2085d84, State: Initialized, Role: FOLLOWER
I20260926 11:43:13.775300 22217 heartbeater.cc:503] Master 127.6.96.62:45691 was elected leader, sending a full tablet report...
I20260926 11:43:13.775365 22370 consensus_queue.cc:260] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:43:13.775367 22369 consensus_queue.cc:260] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:43:13.775487 22368 ts_tablet_manager.cc:1428] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 520628b8ec3e40a6837bd876ebc09433: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260926 11:43:13.776255 22369 ts_tablet_manager.cc:1428] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260926 11:43:13.776293 22370 ts_tablet_manager.cc:1428] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260926 11:43:13.776561 22083 heartbeater.cc:503] Master 127.6.96.62:45691 was elected leader, sending a full tablet report...
I20260926 11:43:13.776947 22351 heartbeater.cc:503] Master 127.6.96.62:45691 was elected leader, sending a full tablet report...
W20260926 11:43:13.779306 22084 tablet.cc:2369] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:43:13.824941 22376 raft_consensus.cc:482] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:43:13.825155 22376 raft_consensus.cc:504] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:43:13.826400 22376 leader_election.cc:288] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 520628b8ec3e40a6837bd876ebc09433 (127.6.96.2:44021), 38b24333651147528537f839a2085d84 (127.6.96.1:44571)
I20260926 11:43:13.831164 22366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:13.826743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:46917 (local address 127.6.96.1:44571)
0926 11:43:13.826882 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:13.826887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:13.827143 (+ 256us) server_negotiation.cc:407] Connection header received
0926 11:43:13.827221 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:13.827223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:13.827269 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:13.827349 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:43:13.827898 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.828622 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.830204 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.830621 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.830713 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:13.830776 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 11:43:13.830983 (+ 207us) server_negotiation.cc:299] Negotiation successful
0926 11:43:13.831053 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 11:43:13.831285 22353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:13.826633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:44021 (local address 127.6.96.3:48697)
0926 11:43:13.826786 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:13.826801 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:43:13.826915 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:43:13.827407 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:43:13.827410 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:43:13.827429 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:43:13.827536 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:13.827541 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.828612 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.828615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:13.830057 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:13.830063 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.830784 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.830788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:13.830959 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:13.830999 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 11:43:13.831071 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 11:43:13.831144 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":53}
I20260926 11:43:13.831269 22363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:13.827018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:48697 (local address 127.6.96.2:44021)
0926 11:43:13.827139 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:13.827143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:13.827157 (+ 14us) server_negotiation.cc:407] Connection header received
0926 11:43:13.827209 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:13.827212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:13.827269 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:13.827351 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:43:13.827685 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.828497 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.830214 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.830662 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.830765 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:13.830851 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 11:43:13.831089 (+ 238us) server_negotiation.cc:299] Negotiation successful
0926 11:43:13.831159 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 11:43:13.831187 22377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:13.826646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:44571 (local address 127.6.96.3:46917)
0926 11:43:13.827052 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:13.827063 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 11:43:13.827151 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:43:13.827656 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:43:13.827658 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:43:13.827669 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:43:13.827743 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:13.827746 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.828727 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.828741 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:13.830046 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:13.830057 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.830739 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.830742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:13.830862 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:13.830886 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 11:43:13.830954 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 11:43:13.831027 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":59,"thread_start_us":155,"threads_started":1}
I20260926 11:43:13.831889 22171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e7d1f44c3b04f6a86c88cb3e6a9ffc3" candidate_uuid: "6afe305187804e80882f2700bdd14972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "520628b8ec3e40a6837bd876ebc09433" is_pre_election: true
I20260926 11:43:13.832028 22037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e7d1f44c3b04f6a86c88cb3e6a9ffc3" candidate_uuid: "6afe305187804e80882f2700bdd14972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b24333651147528537f839a2085d84" is_pre_election: true
I20260926 11:43:13.832201 22171 raft_consensus.cc:2454] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 520628b8ec3e40a6837bd876ebc09433 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6afe305187804e80882f2700bdd14972 in term 0.
I20260926 11:43:13.832297 22037 raft_consensus.cc:2454] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6afe305187804e80882f2700bdd14972 in term 0.
I20260926 11:43:13.832677 22241 leader_election.cc:302] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 38b24333651147528537f839a2085d84, 520628b8ec3e40a6837bd876ebc09433, 6afe305187804e80882f2700bdd14972; no voters:
I20260926 11:43:13.832948 22376 raft_consensus.cc:2781] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:43:13.833025 22376 raft_consensus.cc:482] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:43:13.833056 22376 raft_consensus.cc:3037] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:43:13.834050 22376 raft_consensus.cc:504] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:43:13.834388 22376 leader_election.cc:288] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972 [CANDIDATE]: Term 1 election: Requested vote from peers 520628b8ec3e40a6837bd876ebc09433 (127.6.96.2:44021), 38b24333651147528537f839a2085d84 (127.6.96.1:44571)
I20260926 11:43:13.834765 22037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e7d1f44c3b04f6a86c88cb3e6a9ffc3" candidate_uuid: "6afe305187804e80882f2700bdd14972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b24333651147528537f839a2085d84"
I20260926 11:43:13.834916 22037 raft_consensus.cc:3037] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:43:13.835902 22037 raft_consensus.cc:2454] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6afe305187804e80882f2700bdd14972 in term 1.
I20260926 11:43:13.836165 22171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e7d1f44c3b04f6a86c88cb3e6a9ffc3" candidate_uuid: "6afe305187804e80882f2700bdd14972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "520628b8ec3e40a6837bd876ebc09433"
I20260926 11:43:13.836208 22238 leader_election.cc:302] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b24333651147528537f839a2085d84, 6afe305187804e80882f2700bdd14972; no voters:
I20260926 11:43:13.836299 22171 raft_consensus.cc:3037] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 520628b8ec3e40a6837bd876ebc09433 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:43:13.836364 22376 raft_consensus.cc:2781] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:43:13.836627 22376 raft_consensus.cc:686] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972 [term 1 LEADER]: Becoming Leader. State: Replica: 6afe305187804e80882f2700bdd14972, State: Running, Role: LEADER
I20260926 11:43:13.836992 22376 consensus_queue.cc:237] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:43:13.837491 22171 raft_consensus.cc:2454] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 520628b8ec3e40a6837bd876ebc09433 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6afe305187804e80882f2700bdd14972 in term 1.
I20260926 11:43:13.840199 21892 catalog_manager.cc:5495] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6afe305187804e80882f2700bdd14972 (127.6.96.3). New cstate: current_term: 1 leader_uuid: "6afe305187804e80882f2700bdd14972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } health_report { overall_health: UNKNOWN } } }
I20260926 11:43:13.852592 22364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:13.848808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41450 (local address 127.6.96.3:34785)
0926 11:43:13.848953 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:13.848957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:13.848999 (+ 42us) server_negotiation.cc:407] Connection header received
0926 11:43:13.849175 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:13.849178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:13.849233 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:13.849320 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:43:13.849543 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.850093 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.851099 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:13.851312 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:13.851428 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:13.851724 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:43:13.852174 (+ 450us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:43:13.852263 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 11:43:13.852362 (+ 99us) server_negotiation.cc:299] Negotiation successful
0926 11:43:13.852419 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 11:43:13.858536 22037 raft_consensus.cc:1261] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Refusing update from remote peer 6afe305187804e80882f2700bdd14972: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:43:13.858536 22171 raft_consensus.cc:1261] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 520628b8ec3e40a6837bd876ebc09433 [term 1 FOLLOWER]: Refusing update from remote peer 6afe305187804e80882f2700bdd14972: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:43:13.859329 22376 consensus_queue.cc:1035] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:43:13.859540 22378 consensus_queue.cc:1035] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:43:13.866021 22382 mvcc.cc:205] Tried to move back new op lower bound from 7333572582837084160 to 7333572582759464960. Current Snapshot: MvccSnapshot[applied={T|T < 7333572582837084160}]
I20260926 11:43:13.867733 22383 mvcc.cc:205] Tried to move back new op lower bound from 7333572582837084160 to 7333572582759464960. Current Snapshot: MvccSnapshot[applied={T|T < 7333572582837084160}]
I20260926 11:43:13.875747 21891 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:43:13.875911 21891 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:43:13.878212 21891 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9: Sending DeleteTablet for 3 replicas of tablet 5e7d1f44c3b04f6a86c88cb3e6a9ffc3
I20260926 11:43:13.879103 22151 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:43:13 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:36100
I20260926 11:43:13.879122 22285 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:43:13 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:41446
I20260926 11:43:13.879591 22391 tablet_replica.cc:330] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972: stopping tablet replica
I20260926 11:43:13.879629 22017 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:43:13 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:52624
I20260926 11:43:13.879908 22391 raft_consensus.cc:2229] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972 [term 1 LEADER]: Raft consensus shutting down.
I20260926 11:43:13.880167 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 21954
I20260926 11:43:13.880333 22391 raft_consensus.cc:2258] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:43:13.881875 22391 ts_tablet_manager.cc:1905] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:43:13.883646 22392 tablet_replica.cc:330] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 520628b8ec3e40a6837bd876ebc09433: stopping tablet replica
I20260926 11:43:13.883987 22392 raft_consensus.cc:2229] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 520628b8ec3e40a6837bd876ebc09433 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:43:13.884336 22392 raft_consensus.cc:2258] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 520628b8ec3e40a6837bd876ebc09433 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:43:13.885063 22391 ts_tablet_manager.cc:1918] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:43:13.885164 22391 log.cc:1196] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/5e7d1f44c3b04f6a86c88cb3e6a9ffc3
I20260926 11:43:13.885510 22391 ts_tablet_manager.cc:1939] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 6afe305187804e80882f2700bdd14972: Deleting consensus metadata
I20260926 11:43:13.886267 22392 ts_tablet_manager.cc:1905] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 520628b8ec3e40a6837bd876ebc09433: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:43:13.886762 21876 catalog_manager.cc:4841] TS 6afe305187804e80882f2700bdd14972 (127.6.96.3:34785): tablet 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 (table test-table [id=b328afa742ef454fbce959199899e9d9]) successfully deleted
W20260926 11:43:13.888492 21876 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.96.1:44571 (error 108)
I20260926 11:43:13.888742 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 22088
I20260926 11:43:13.890676 22367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:13.890378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:44571 (local address 127.0.0.1:52634)
0926 11:43:13.890518 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:13.890587 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.1:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260926 11:43:13.890960 21876 catalog_manager.cc:4584] TS 38b24333651147528537f839a2085d84 (127.6.96.1:44571): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5e7d1f44c3b04f6a86c88cb3e6a9ffc3: Network error: Client connection negotiation failed: client connection to 127.6.96.1:44571: connect: Connection refused (error 111)
I20260926 11:43:13.893882 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 22222
I20260926 11:43:13.895855 22367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:13.895571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:44021 (local address 127.0.0.1:36116)
0926 11:43:13.895699 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:13.895766 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.2:44021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260926 11:43:13.898662 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 21860
I20260926 11:43:33.904042 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:45691
--webserver_interface=127.6.96.62
--webserver_port=36629
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:45271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:45691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:43:34.013652 22395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:43:34.013928 22395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:43:34.013969 22395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:43:34.018042 22395 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 11:43:34.018114 22395 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 11:43:34.018131 22395 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 11:43:34.018142 22395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:43:34.018159 22395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:43:34.018175 22395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:43:34.018195 22395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:43:34.018224 22395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:43:34.023692 22395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:45271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:45691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:45691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=36629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:43:34.024791 22395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:43:34.025902 22395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:43:34.039320 22401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:43:34.039333 22402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:43:34.039382 22395 server_base.cc:1034] running on GCE node
W20260926 11:43:34.039318 22404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:43:34.040314 22395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:43:34.041510 22395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:43:34.042683 22395 hybrid_clock.cc:648] HybridClock initialized: now 1790423014042653 us; error 46 us; skew 500 ppm
Sep 26 11:43:34 dist-test-slave-xxxx krb5kdc[21847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423014, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:43:34.052486 22395 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:43:34.053571 22395 webserver.cc:460] Webserver started at http://127.6.96.62:36629/ using document root <none> and password file <none>
I20260926 11:43:34.054183 22395 fs_manager.cc:359] Metadata directory not provided
I20260926 11:43:34.054275 22395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:43:34.058535 22395 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:43:34.060945 22411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:34.062276 22395 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 11:43:34.062417 22395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "6395ba7907354546ae2ae6772a73a5d9"
format_stamp: "Formatted at 2026-09-26 11:43:12 on dist-test-slave-xxxx"
server_key: "481ac7e305a6400a47feaff7f8c77813"
server_key_iv: "489fec93914a580bc666f848124e9627"
server_key_version: "encryptionkey@0"
I20260926 11:43:34.063149 22395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:43:34.086673 22395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:43:34.089882 22395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:43:34.090066 22395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:43:34.097124 22395 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:45691
I20260926 11:43:34.097399 22463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:45691 every 8 connection(s)
I20260926 11:43:34.098217 22395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:43:34.100689 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 22395
I20260926 11:43:34.101115 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:44571
--local_ip_for_outbound_sockets=127.6.96.1
--tserver_master_addrs=127.6.96.62:45691
--webserver_port=46711
--webserver_interface=127.6.96.1
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:45271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 11:43:34.102712 22464 sys_catalog.cc:263] Verifying existing consensus state
I20260926 11:43:34.104518 22464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9: Bootstrap starting.
I20260926 11:43:34.121377 22464 log.cc:824] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9: Log is configured to *not* fsync() on all Append() calls
I20260926 11:43:34.130524 22464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 11:43:34.131054 22464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9: Bootstrap complete.
I20260926 11:43:34.135108 22464 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6395ba7907354546ae2ae6772a73a5d9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45691 } }
I20260926 11:43:34.135561 22464 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6395ba7907354546ae2ae6772a73a5d9, State: Initialized, Role: FOLLOWER
I20260926 11:43:34.136093 22464 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6395ba7907354546ae2ae6772a73a5d9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45691 } }
I20260926 11:43:34.136224 22464 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:43:34.136302 22464 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:43:34.136399 22464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 1 FOLLOWER]: Advancing to term 2
I20260926 11:43:34.138468 22464 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6395ba7907354546ae2ae6772a73a5d9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45691 } }
I20260926 11:43:34.138862 22464 leader_election.cc:302] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6395ba7907354546ae2ae6772a73a5d9; no voters:
I20260926 11:43:34.139204 22464 leader_election.cc:288] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 11:43:34.139353 22468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 11:43:34.139614 22468 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 2 LEADER]: Becoming Leader. State: Replica: 6395ba7907354546ae2ae6772a73a5d9, State: Running, Role: LEADER
I20260926 11:43:34.139993 22468 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6395ba7907354546ae2ae6772a73a5d9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45691 } }
I20260926 11:43:34.140483 22464 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:43:34.141450 22469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6395ba7907354546ae2ae6772a73a5d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6395ba7907354546ae2ae6772a73a5d9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45691 } } }
I20260926 11:43:34.141574 22469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [sys.catalog]: This master's current role is: LEADER
I20260926 11:43:34.142495 22470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6395ba7907354546ae2ae6772a73a5d9. Latest consensus state: current_term: 2 leader_uuid: "6395ba7907354546ae2ae6772a73a5d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6395ba7907354546ae2ae6772a73a5d9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45691 } } }
I20260926 11:43:34.142928 22470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [sys.catalog]: This master's current role is: LEADER
I20260926 11:43:34.145267 22484 catalog_manager.cc:1259] Loaded cluster ID: 8eae91f9b435482a8e9d2f9d9dde6031
I20260926 11:43:34.145321 22484 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9: loading cluster ID for follower catalog manager: success
I20260926 11:43:34.147214 22484 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9: acquiring CA information for follower catalog manager: success
I20260926 11:43:34.148474 22484 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 11:43:34.148602 22477 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:43:34.151007 22477 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:43:34.151467 22477 catalog_manager.cc:1259] Loaded cluster ID: 8eae91f9b435482a8e9d2f9d9dde6031
I20260926 11:43:34.151522 22477 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:43:34.152163 22477 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:43:34.152604 22477 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9: Loaded TSK: 0
I20260926 11:43:34.164767 22477 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9: Generated new TSK 1
I20260926 11:43:34.165040 22477 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 11:43:34.221390 22466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:43:34.221697 22466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:43:34.221743 22466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:43:34.226022 22466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:43:34.226097 22466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:43:34.226166 22466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:43:34.231665 22466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:45271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:44571
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=46711
--tserver_master_addrs=127.6.96.62:45691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:43:34.232736 22466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:43:34.233789 22466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:43:34.243856 22491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:43:34.243794 22492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:43:34.243798 22494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:43:34.244094 22466 server_base.cc:1034] running on GCE node
I20260926 11:43:34.244611 22466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:43:34.245285 22466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:43:34.246479 22466 hybrid_clock.cc:648] HybridClock initialized: now 1790423014246438 us; error 55 us; skew 500 ppm
Sep 26 11:43:34 dist-test-slave-xxxx krb5kdc[21847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423014, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:43:34.249188 22466 init.cc:377] Logged in from keytab as kudu/127.6.96.1@KRBTEST.COM (short username kudu)
I20260926 11:43:34.250128 22466 webserver.cc:460] Webserver started at http://127.6.96.1:46711/ using document root <none> and password file <none>
I20260926 11:43:34.250663 22466 fs_manager.cc:359] Metadata directory not provided
I20260926 11:43:34.250732 22466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:43:34.254585 22466 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:43:34.256731 22501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:34.258044 22466 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 11:43:34.258150 22466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "38b24333651147528537f839a2085d84"
format_stamp: "Formatted at 2026-09-26 11:43:13 on dist-test-slave-xxxx"
server_key: "324a2e58deb6fe391d9743567b8da01a"
server_key_iv: "5ed8a67423014a80c6574189d1c82c49"
server_key_version: "encryptionkey@0"
I20260926 11:43:34.258724 22466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:43:34.284847 22466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:43:34.288028 22466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:43:34.288204 22466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:43:34.288834 22466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:43:34.289922 22508 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 11:43:34.293933 22466 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 11:43:34.294004 22466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260926 11:43:34.294059 22466 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 11:43:34.295835 22466 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 11:43:34.295879 22466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.001s
I20260926 11:43:34.296150 22508 tablet_bootstrap.cc:492] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84: Bootstrap starting.
I20260926 11:43:34.305511 22466 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:44571
I20260926 11:43:34.305574 22615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:44571 every 8 connection(s)
I20260926 11:43:34.306470 22466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:43:34.307621 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 22466
I20260926 11:43:34.308013 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:44021
--local_ip_for_outbound_sockets=127.6.96.2
--tserver_master_addrs=127.6.96.62:45691
--webserver_port=43043
--webserver_interface=127.6.96.2
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:45271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 11:43:34.314594 22508 log.cc:824] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84: Log is configured to *not* fsync() on all Append() calls
Sep 26 11:43:34 dist-test-slave-xxxx krb5kdc[21847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423014, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:43:34.323448 22619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:34.309013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:43239 (local address 127.6.96.62:45691)
0926 11:43:34.309579 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:34.309612 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:34.310132 (+ 520us) server_negotiation.cc:407] Connection header received
0926 11:43:34.311636 (+ 1504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:34.311649 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:34.311973 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:34.312295 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:43:34.313494 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.314473 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.315909 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.316375 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.316522 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:34.318776 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:43:34.318781 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:43:34.318792 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:43:34.318818 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:43:34.320338 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:34.320808 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:34.320812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:34.320816 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:34.320929 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:34.321235 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:34.321239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:34.321240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:34.321587 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:43:34.321702 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 11:43:34.322233 (+ 531us) server_negotiation.cc:299] Negotiation successful
0926 11:43:34.322484 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":139,"threads_started":1}
I20260926 11:43:34.323467 22508 tablet_bootstrap.cc:492] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 11:43:34.323460 22618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:34.308860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.1:43239)
0926 11:43:34.309924 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:34.309982 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 11:43:34.311147 (+ 1165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:43:34.312373 (+ 1226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:43:34.312381 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:43:34.312803 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:43:34.313326 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:34.313343 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.314629 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.314632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:34.315796 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:34.315806 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.316511 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.316515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:34.316694 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:34.317228 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:43:34.317244 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:43:34.318597 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:43:34.320447 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:43:34.320453 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:43:34.320465 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:43:34.320704 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:43:34.321015 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:43:34.321018 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:43:34.321020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:43:34.321152 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:43:34.321691 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:43:34.321698 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:43:34.321942 (+ 244us) client_negotiation.cc:769] Sending connection context
0926 11:43:34.322107 (+ 165us) client_negotiation.cc:240] Negotiation successful
0926 11:43:34.322445 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"thread_start_us":143,"threads_started":1}
I20260926 11:43:34.323961 22508 tablet_bootstrap.cc:492] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84: Bootstrap complete.
I20260926 11:43:34.326301 22508 ts_tablet_manager.cc:1397] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84: Time spent bootstrapping tablet: real 0.030s user 0.020s sys 0.008s
I20260926 11:43:34.326464 22616 heartbeater.cc:347] Connected to a master server at 127.6.96.62:45691
I20260926 11:43:34.326694 22616 heartbeater.cc:464] Registering TS with master...
I20260926 11:43:34.327296 22616 heartbeater.cc:511] Master 127.6.96.62:45691 requested a full tablet report, sending...
I20260926 11:43:34.329200 22508 raft_consensus.cc:348] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:43:34.329619 22508 raft_consensus.cc:729] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b24333651147528537f839a2085d84, State: Initialized, Role: FOLLOWER
I20260926 11:43:34.330075 22508 consensus_queue.cc:260] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:43:34.330084 22428 ts_manager.cc:194] Registered new tserver with Master: 38b24333651147528537f839a2085d84 (127.6.96.1:44571)
I20260926 11:43:34.331001 22508 ts_tablet_manager.cc:1428] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260926 11:43:34.334451 22428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.1@KRBTEST.COM'} at 127.6.96.1:43239
I20260926 11:43:34.337186 22616 heartbeater.cc:503] Master 127.6.96.62:45691 was elected leader, sending a full tablet report...
Sep 26 11:43:34 dist-test-slave-xxxx krb5kdc[21847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423014, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for kudu/127.6.96.1@KRBTEST.COM
I20260926 11:43:34.343955 22624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:34.333496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40888 (local address 127.6.96.1:44571)
0926 11:43:34.334273 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:34.334282 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:34.334300 (+ 18us) server_negotiation.cc:407] Connection header received
0926 11:43:34.334355 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:34.334359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:34.334528 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:34.335324 (+ 796us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:43:34.335434 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.336402 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.337564 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.338013 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.338117 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:34.340590 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:43:34.340599 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:43:34.340608 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:43:34.340630 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:43:34.342144 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:34.342633 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:34.342637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:34.342639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:34.342716 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:34.343016 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:34.343019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:34.343020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:34.343243 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:43:34.343338 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 11:43:34.343681 (+ 343us) server_negotiation.cc:299] Negotiation successful
0926 11:43:34.343823 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":695,"thread_start_us":93,"threads_started":1}
I20260926 11:43:34.343948 22623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:34.333443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:44571 (local address 127.0.0.1:40888)
0926 11:43:34.333848 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:34.333892 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 11:43:34.334020 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:43:34.334690 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:43:34.334695 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:43:34.335064 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:43:34.335171 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:34.335178 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.336534 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.336538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:34.337452 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:34.337464 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.338117 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.338134 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:34.338242 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:34.338806 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:43:34.338820 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:43:34.340462 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:43:34.342226 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:43:34.342233 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:43:34.342238 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:43:34.342534 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:43:34.342810 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:43:34.342812 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:43:34.342815 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:43:34.342924 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:43:34.343336 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:43:34.343341 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:43:34.343463 (+ 122us) client_negotiation.cc:769] Sending connection context
0926 11:43:34.343595 (+ 132us) client_negotiation.cc:240] Negotiation successful
0926 11:43:34.343780 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":135,"threads_started":1}
I20260926 11:43:34.345000 22550 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:43:13 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:40888
I20260926 11:43:34.345832 22625 tablet_replica.cc:330] stopping tablet replica
I20260926 11:43:34.346154 22625 raft_consensus.cc:2229] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:43:34.346408 22625 raft_consensus.cc:2258] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:43:34.347582 22625 ts_tablet_manager.cc:1905] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:43:34.352128 22625 ts_tablet_manager.cc:1918] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:43:34.352231 22625 log.cc:1196] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/wal/wals/5e7d1f44c3b04f6a86c88cb3e6a9ffc3
I20260926 11:43:34.352491 22625 ts_tablet_manager.cc:1939] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 38b24333651147528537f839a2085d84: Deleting consensus metadata
I20260926 11:43:34.354477 22412 catalog_manager.cc:4841] TS 38b24333651147528537f839a2085d84 (127.6.96.1:44571): tablet 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 (table test-table [id=b328afa742ef454fbce959199899e9d9]) successfully deleted
W20260926 11:43:34.431607 22620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:43:34.431874 22620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:43:34.431917 22620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:43:34.435997 22620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:43:34.436049 22620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:43:34.436107 22620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:43:34.441601 22620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:45271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:44021
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=43043
--tserver_master_addrs=127.6.96.62:45691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:43:34.442662 22620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:43:34.443691 22620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:43:34.453166 22630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:43:34.453239 22620 server_base.cc:1034] running on GCE node
W20260926 11:43:34.453294 22633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:43:34.453519 22631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:43:34.453954 22620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:43:34.454622 22620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:43:34.455773 22620 hybrid_clock.cc:648] HybridClock initialized: now 1790423014455739 us; error 45 us; skew 500 ppm
Sep 26 11:43:34 dist-test-slave-xxxx krb5kdc[21847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423014, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:43:34.458372 22620 init.cc:377] Logged in from keytab as kudu/127.6.96.2@KRBTEST.COM (short username kudu)
I20260926 11:43:34.459246 22620 webserver.cc:460] Webserver started at http://127.6.96.2:43043/ using document root <none> and password file <none>
I20260926 11:43:34.459758 22620 fs_manager.cc:359] Metadata directory not provided
I20260926 11:43:34.459824 22620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:43:34.463675 22620 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 11:43:34.465703 22640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:34.467010 22620 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:43:34.467121 22620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "520628b8ec3e40a6837bd876ebc09433"
format_stamp: "Formatted at 2026-09-26 11:43:13 on dist-test-slave-xxxx"
server_key: "ba1e388515aa42c7838d86eb783e303e"
server_key_iv: "451a289daf1c4f746f90ce8b7d821cdb"
server_key_version: "encryptionkey@0"
I20260926 11:43:34.467815 22620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:43:34.489449 22620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:43:34.492632 22620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:43:34.492808 22620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:43:34.493376 22620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:43:34.494395 22647 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260926 11:43:34.498299 22647 ts_tablet_manager.cc:1852] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 520628b8ec3e40a6837bd876ebc09433: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260926 11:43:34.498751 22647 ts_tablet_manager.cc:1905] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 520628b8ec3e40a6837bd876ebc09433: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:43:34.502784 22647 ts_tablet_manager.cc:1918] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 520628b8ec3e40a6837bd876ebc09433: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:43:34.502916 22647 log.cc:1196] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 520628b8ec3e40a6837bd876ebc09433: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/wal/wals/5e7d1f44c3b04f6a86c88cb3e6a9ffc3
I20260926 11:43:34.503237 22647 ts_tablet_manager.cc:1939] T 5e7d1f44c3b04f6a86c88cb3e6a9ffc3 P 520628b8ec3e40a6837bd876ebc09433: Deleting consensus metadata
I20260926 11:43:34.503511 22620 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260926 11:43:34.503571 22620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.009s user 0.000s sys 0.000s
I20260926 11:43:34.503654 22620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:43:34.503683 22620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:34.512346 22620 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:44021
I20260926 11:43:34.512374 22754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:44021 every 8 connection(s)
I20260926 11:43:34.513211 22620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:43:34.516305 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 22620
I20260926 11:43:34.516770 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:34785
--local_ip_for_outbound_sockets=127.6.96.3
--tserver_master_addrs=127.6.96.62:45691
--webserver_port=35027
--webserver_interface=127.6.96.3
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:45271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 11:43:34 dist-test-slave-xxxx krb5kdc[21847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423014, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:43:34.527848 22619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:34.514950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:37481 (local address 127.6.96.62:45691)
0926 11:43:34.515123 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:34.515128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:34.516043 (+ 915us) server_negotiation.cc:407] Connection header received
0926 11:43:34.517212 (+ 1169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:34.517215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:34.517268 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:34.517344 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:43:34.518776 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.519485 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.520638 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.521009 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.521136 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:34.523766 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:43:34.523771 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:43:34.523774 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:43:34.523808 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:43:34.525647 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:34.526188 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:34.526192 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:34.526194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:34.526287 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:34.526660 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:34.526663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:34.526665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:34.526929 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:43:34.527033 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 11:43:34.527536 (+ 503us) server_negotiation.cc:299] Negotiation successful
0926 11:43:34.527674 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 11:43:34.528697 22757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:34.515421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.2:37481)
0926 11:43:34.515885 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:34.515928 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 11:43:34.516974 (+ 1046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:43:34.517583 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:43:34.517592 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:43:34.518051 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:43:34.518622 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:34.518636 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.519628 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.519631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:34.520508 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:34.520517 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.521156 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.521158 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:34.521302 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:34.521928 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:43:34.521944 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:43:34.523581 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:43:34.525794 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:43:34.525799 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:43:34.525814 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:43:34.526092 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:43:34.526408 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:43:34.526412 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:43:34.526414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:43:34.526563 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:43:34.527054 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:43:34.527060 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:43:34.527306 (+ 246us) client_negotiation.cc:769] Sending connection context
0926 11:43:34.527476 (+ 170us) client_negotiation.cc:240] Negotiation successful
0926 11:43:34.527744 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":113,"threads_started":1}
I20260926 11:43:34.529863 22755 heartbeater.cc:347] Connected to a master server at 127.6.96.62:45691
I20260926 11:43:34.530148 22755 heartbeater.cc:464] Registering TS with master...
I20260926 11:43:34.530758 22755 heartbeater.cc:511] Master 127.6.96.62:45691 requested a full tablet report, sending...
I20260926 11:43:34.531929 22428 ts_manager.cc:194] Registered new tserver with Master: 520628b8ec3e40a6837bd876ebc09433 (127.6.96.2:44021)
I20260926 11:43:34.532500 22428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.2@KRBTEST.COM'} at 127.6.96.2:37481
W20260926 11:43:34.629987 22758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:43:34.630255 22758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:43:34.630302 22758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:43:34.634434 22758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:43:34.634496 22758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:43:34.634564 22758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:43:34.640179 22758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:45271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:34785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=35027
--tserver_master_addrs=127.6.96.62:45691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:43:34.641330 22758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:43:34.642453 22758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:43:34.652173 22763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:43:34.652103 22764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:43:34.652139 22758 server_base.cc:1034] running on GCE node
W20260926 11:43:34.652299 22766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:43:34.652685 22758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:43:34.653340 22758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:43:34.654531 22758 hybrid_clock.cc:648] HybridClock initialized: now 1790423014654515 us; error 32 us; skew 500 ppm
Sep 26 11:43:34 dist-test-slave-xxxx krb5kdc[21847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423014, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:43:34.657328 22758 init.cc:377] Logged in from keytab as kudu/127.6.96.3@KRBTEST.COM (short username kudu)
I20260926 11:43:34.658275 22758 webserver.cc:460] Webserver started at http://127.6.96.3:35027/ using document root <none> and password file <none>
I20260926 11:43:34.658810 22758 fs_manager.cc:359] Metadata directory not provided
I20260926 11:43:34.658927 22758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:43:34.662876 22758 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:43:34.665158 22773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:34.666638 22758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 11:43:34.666743 22758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "6afe305187804e80882f2700bdd14972"
format_stamp: "Formatted at 2026-09-26 11:43:13 on dist-test-slave-xxxx"
server_key: "29f55bcb297d9fe96efe4145e68c4fa0"
server_key_iv: "4885cf12e371264cab2599288556efd1"
server_key_version: "encryptionkey@0"
I20260926 11:43:34.667368 22758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:43:34.683415 22758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:43:34.686564 22758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:43:34.686741 22758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:43:34.687391 22758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:43:34.688227 22758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:43:34.688282 22758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:34.688335 22758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:43:34.688359 22758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:34.697117 22758 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:34785
I20260926 11:43:34.697148 22886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:34785 every 8 connection(s)
I20260926 11:43:34.698093 22758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:43:34.703769 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 22758
Sep 26 11:43:34 dist-test-slave-xxxx krb5kdc[21847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423014, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:43:34.712013 22619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:34.699901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:56099 (local address 127.6.96.62:45691)
0926 11:43:34.700065 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:34.700069 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:34.701084 (+ 1015us) server_negotiation.cc:407] Connection header received
0926 11:43:34.701918 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:34.701920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:34.701968 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:34.702043 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:43:34.703235 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.703901 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.705368 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.705612 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.705695 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:34.708480 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:43:34.708483 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:43:34.708485 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:43:34.708508 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:43:34.709888 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:34.710414 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:34.710419 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:34.710420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:34.710512 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:34.710867 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:34.710871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:34.710872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:34.711068 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:43:34.711182 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 11:43:34.711726 (+ 544us) server_negotiation.cc:299] Negotiation successful
0926 11:43:34.711835 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62,"spinlock_wait_cycles":4352}
I20260926 11:43:34.712775 22891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:34.704422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47972 (local address 127.6.96.62:45691)
0926 11:43:34.704659 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:34.704662 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:34.704825 (+ 163us) server_negotiation.cc:407] Connection header received
0926 11:43:34.704927 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:34.704929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:34.704971 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:34.705055 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:43:34.705893 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.706545 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.707683 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.707916 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.708015 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:34.709892 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:43:34.709895 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:43:34.709897 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:43:34.709919 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:43:34.711315 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:34.711686 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:34.711689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:34.711691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:34.711774 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:34.712015 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:34.712018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:34.712020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:34.712216 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:43:34.712410 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0926 11:43:34.712561 (+ 151us) server_negotiation.cc:299] Negotiation successful
0926 11:43:34.712654 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"spinlock_wait_cycles":1408,"thread_start_us":61,"threads_started":1}
I20260926 11:43:34.713032 22889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:34.700296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.3:56099)
0926 11:43:34.700931 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:34.700972 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 11:43:34.701733 (+ 761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:43:34.702209 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:43:34.702215 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:43:34.702605 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:43:34.703113 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:34.703127 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.704035 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.704038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:34.705243 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:34.705254 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.705774 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.705779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:34.705978 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:34.706617 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:43:34.706644 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:43:34.708325 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:43:34.709992 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:43:34.709998 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:43:34.710015 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:43:34.710303 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:43:34.710600 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:43:34.710603 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:43:34.710605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:43:34.710758 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:43:34.711166 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:43:34.711173 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:43:34.711491 (+ 318us) client_negotiation.cc:769] Sending connection context
0926 11:43:34.711656 (+ 165us) client_negotiation.cc:240] Negotiation successful
0926 11:43:34.711991 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":107,"threads_started":1}
I20260926 11:43:34.714147 22887 heartbeater.cc:347] Connected to a master server at 127.6.96.62:45691
I20260926 11:43:34.714394 22887 heartbeater.cc:464] Registering TS with master...
I20260926 11:43:34.714897 22887 heartbeater.cc:511] Master 127.6.96.62:45691 requested a full tablet report, sending...
I20260926 11:43:34.715821 22426 ts_manager.cc:194] Registered new tserver with Master: 6afe305187804e80882f2700bdd14972 (127.6.96.3:34785)
I20260926 11:43:34.716537 22426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.3@KRBTEST.COM'} at 127.6.96.3:56099
I20260926 11:43:34.726531 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 11:43:34.730676 22891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:34.727136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47988 (local address 127.6.96.62:45691)
0926 11:43:34.727279 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:34.727284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:34.727513 (+ 229us) server_negotiation.cc:407] Connection header received
0926 11:43:34.727696 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:34.727699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:34.727808 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:34.727899 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:43:34.728149 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.728680 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.729544 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.729775 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.729859 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:34.730157 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:43:34.730294 (+ 137us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0926 11:43:34.730545 (+ 251us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47988: token expired
Metrics: {"server-negotiator.queue_time_us":52}
W20260926 11:43:34.730746 22891 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47988: token expired
W20260926 11:43:34.731077 22355 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.6.96.62:45691)
I20260926 11:43:34.737978 22891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:34.731474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47992 (local address 127.6.96.62:45691)
0926 11:43:34.731585 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:34.731589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:34.731601 (+ 12us) server_negotiation.cc:407] Connection header received
0926 11:43:34.731632 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:34.731634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:34.731671 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:34.731729 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:43:34.732298 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.732768 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.733547 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.733751 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.733832 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:34.735125 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:43:34.735128 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:43:34.735131 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:43:34.735157 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:43:34.736496 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:34.736877 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:34.736881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:34.736882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:34.736965 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:34.737196 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:34.737199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:34.737201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:34.737369 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:43:34.737480 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 11:43:34.737700 (+ 220us) server_negotiation.cc:299] Negotiation successful
0926 11:43:34.737810 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 11:43:34.740644 22426 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47992:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 11:43:34.744237 22426 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:43:34.755362 22550 tablet_service.cc:1467] Processing CreateTablet for tablet 8f38c941dbd446a7867ba76c9458f9bd (DEFAULT_TABLE table=test-table [id=80151774064948a9b67dbc8af1b9a225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:43:34.756011 22550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f38c941dbd446a7867ba76c9458f9bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:43:34.758356 22508 tablet_bootstrap.cc:492] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84: Bootstrap starting.
I20260926 11:43:34.759857 22508 tablet_bootstrap.cc:654] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84: Neither blocks nor log segments found. Creating new log.
I20260926 11:43:34.761199 22508 tablet_bootstrap.cc:492] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84: No bootstrap required, opened a new log
I20260926 11:43:34.761291 22508 ts_tablet_manager.cc:1397] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260926 11:43:34.761785 22508 raft_consensus.cc:348] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:43:34.761897 22508 raft_consensus.cc:374] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:43:34.761932 22508 raft_consensus.cc:729] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b24333651147528537f839a2085d84, State: Initialized, Role: FOLLOWER
I20260926 11:43:34.762049 22508 consensus_queue.cc:260] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:43:34.762295 22508 ts_tablet_manager.cc:1428] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 11:43:34.765266 22623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:34.754359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:34785 (local address 127.0.0.1:34526)
0926 11:43:34.754523 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:34.754535 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:43:34.754638 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:43:34.760804 (+ 6166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:43:34.760808 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:43:34.760828 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:43:34.760938 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:34.760946 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.762667 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.762671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:34.763798 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:34.763808 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.764823 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.764826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:34.764935 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:34.764981 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 11:43:34.765050 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 11:43:34.765136 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260926 11:43:34.765306 22897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:34.754848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34526 (local address 127.6.96.3:34785)
0926 11:43:34.760333 (+ 5485us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:34.760347 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:34.760372 (+ 25us) server_negotiation.cc:407] Connection header received
0926 11:43:34.760449 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:34.760454 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:34.760649 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:34.761097 (+ 448us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:43:34.761268 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.762549 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.763940 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.764709 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.764803 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:34.764959 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 11:43:34.765089 (+ 130us) server_negotiation.cc:299] Negotiation successful
0926 11:43:34.765183 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4292,"thread_start_us":156,"threads_started":1}
I20260926 11:43:34.765620 22896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:34.754810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:44021 (local address 127.0.0.1:41262)
0926 11:43:34.755331 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:34.755345 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:43:34.755434 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:43:34.760041 (+ 4607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:43:34.760045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:43:34.760066 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:43:34.760193 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:34.760199 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.761801 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.761804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:34.763868 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:34.763880 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.765185 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.765189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:34.765297 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:34.765321 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 11:43:34.765392 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 11:43:34.765463 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":103,"threads_started":1}
I20260926 11:43:34.766211 22898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:34.755297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41262 (local address 127.6.96.2:44021)
0926 11:43:34.759493 (+ 4196us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:34.759505 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:34.759532 (+ 27us) server_negotiation.cc:407] Connection header received
0926 11:43:34.759615 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:34.759621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:34.759863 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:34.760086 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:43:34.760338 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.761664 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.764011 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.764880 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.764984 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:34.765122 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0926 11:43:34.765940 (+ 818us) server_negotiation.cc:299] Negotiation successful
0926 11:43:34.766048 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4081,"thread_start_us":119,"threads_started":1}
I20260926 11:43:34.767664 22689 tablet_service.cc:1467] Processing CreateTablet for tablet 8f38c941dbd446a7867ba76c9458f9bd (DEFAULT_TABLE table=test-table [id=80151774064948a9b67dbc8af1b9a225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:43:34.768790 22689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f38c941dbd446a7867ba76c9458f9bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:43:34.768518 22821 tablet_service.cc:1467] Processing CreateTablet for tablet 8f38c941dbd446a7867ba76c9458f9bd (DEFAULT_TABLE table=test-table [id=80151774064948a9b67dbc8af1b9a225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:43:34.769567 22821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f38c941dbd446a7867ba76c9458f9bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:43:34.774035 22647 tablet_bootstrap.cc:492] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433: Bootstrap starting.
I20260926 11:43:34.776257 22647 tablet_bootstrap.cc:654] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433: Neither blocks nor log segments found. Creating new log.
I20260926 11:43:34.777107 22647 log.cc:824] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433: Log is configured to *not* fsync() on all Append() calls
I20260926 11:43:34.777575 22900 tablet_bootstrap.cc:492] T 8f38c941dbd446a7867ba76c9458f9bd P 6afe305187804e80882f2700bdd14972: Bootstrap starting.
I20260926 11:43:34.780176 22900 tablet_bootstrap.cc:654] T 8f38c941dbd446a7867ba76c9458f9bd P 6afe305187804e80882f2700bdd14972: Neither blocks nor log segments found. Creating new log.
I20260926 11:43:34.781462 22900 log.cc:824] T 8f38c941dbd446a7867ba76c9458f9bd P 6afe305187804e80882f2700bdd14972: Log is configured to *not* fsync() on all Append() calls
I20260926 11:43:34.783354 22900 tablet_bootstrap.cc:492] T 8f38c941dbd446a7867ba76c9458f9bd P 6afe305187804e80882f2700bdd14972: No bootstrap required, opened a new log
I20260926 11:43:34.783617 22900 ts_tablet_manager.cc:1397] T 8f38c941dbd446a7867ba76c9458f9bd P 6afe305187804e80882f2700bdd14972: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260926 11:43:34.783649 22647 tablet_bootstrap.cc:492] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433: No bootstrap required, opened a new log
I20260926 11:43:34.783839 22647 ts_tablet_manager.cc:1397] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.003s
I20260926 11:43:34.786731 22900 raft_consensus.cc:348] T 8f38c941dbd446a7867ba76c9458f9bd P 6afe305187804e80882f2700bdd14972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:43:34.787027 22900 raft_consensus.cc:374] T 8f38c941dbd446a7867ba76c9458f9bd P 6afe305187804e80882f2700bdd14972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:43:34.787084 22900 raft_consensus.cc:729] T 8f38c941dbd446a7867ba76c9458f9bd P 6afe305187804e80882f2700bdd14972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6afe305187804e80882f2700bdd14972, State: Initialized, Role: FOLLOWER
I20260926 11:43:34.787526 22900 consensus_queue.cc:260] T 8f38c941dbd446a7867ba76c9458f9bd P 6afe305187804e80882f2700bdd14972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:43:34.788188 22887 heartbeater.cc:503] Master 127.6.96.62:45691 was elected leader, sending a full tablet report...
I20260926 11:43:34.788057 22647 raft_consensus.cc:348] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:43:34.788328 22647 raft_consensus.cc:374] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:43:34.788380 22647 raft_consensus.cc:729] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 520628b8ec3e40a6837bd876ebc09433, State: Initialized, Role: FOLLOWER
I20260926 11:43:34.788558 22900 ts_tablet_manager.cc:1428] T 8f38c941dbd446a7867ba76c9458f9bd P 6afe305187804e80882f2700bdd14972: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260926 11:43:34.788844 22647 consensus_queue.cc:260] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:43:34.789505 22755 heartbeater.cc:503] Master 127.6.96.62:45691 was elected leader, sending a full tablet report...
I20260926 11:43:34.789882 22647 ts_tablet_manager.cc:1428] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 11:43:34.804147 22904 raft_consensus.cc:482] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:43:34.804368 22904 raft_consensus.cc:504] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:43:34.805545 22904 leader_election.cc:288] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6afe305187804e80882f2700bdd14972 (127.6.96.3:34785), 38b24333651147528537f839a2085d84 (127.6.96.1:44571)
W20260926 11:43:34.808103 22617 tablet.cc:2369] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:43:34.810504 22757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:34.805867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:34785 (local address 127.6.96.2:43619)
0926 11:43:34.806010 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:34.806026 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 11:43:34.806132 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:43:34.806476 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:43:34.806479 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:43:34.806497 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:43:34.806609 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:34.806615 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.807565 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.807568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:34.808954 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:34.808967 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.810040 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.810044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:34.810167 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:34.810193 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 11:43:34.810260 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 11:43:34.810343 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":43}
I20260926 11:43:34.810673 22624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:34.806380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:48963 (local address 127.6.96.1:44571)
0926 11:43:34.806511 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:34.806515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:34.806641 (+ 126us) server_negotiation.cc:407] Connection header received
0926 11:43:34.806759 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:34.806762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:34.806858 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:34.806955 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:43:34.807221 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.807861 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.809298 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.809902 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.809987 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:34.810146 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 11:43:34.810475 (+ 329us) server_negotiation.cc:299] Negotiation successful
0926 11:43:34.810556 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 11:43:34.811025 22905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:34.806151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:44571 (local address 127.6.96.2:48963)
0926 11:43:34.806549 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:34.806563 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:43:34.806671 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:43:34.806972 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:43:34.806975 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:43:34.806992 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:43:34.807116 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:34.807122 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.807963 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.807973 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:34.809159 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:34.809170 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.810241 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.810244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:34.810347 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:34.810367 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 11:43:34.810430 (+ 63us) client_negotiation.cc:240] Negotiation successful
0926 11:43:34.810497 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":23,"thread_start_us":141,"threads_started":1}
I20260926 11:43:34.811017 22897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:34.805900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:43619 (local address 127.6.96.3:34785)
0926 11:43:34.806038 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:34.806042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:34.806079 (+ 37us) server_negotiation.cc:407] Connection header received
0926 11:43:34.806232 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:34.806235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:34.806288 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:34.806356 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:43:34.806734 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.807460 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.809128 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.809806 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.809907 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:34.809976 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 11:43:34.810781 (+ 805us) server_negotiation.cc:299] Negotiation successful
0926 11:43:34.810893 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 11:43:34.811661 22841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f38c941dbd446a7867ba76c9458f9bd" candidate_uuid: "520628b8ec3e40a6837bd876ebc09433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6afe305187804e80882f2700bdd14972" is_pre_election: true
I20260926 11:43:34.811648 22570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f38c941dbd446a7867ba76c9458f9bd" candidate_uuid: "520628b8ec3e40a6837bd876ebc09433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b24333651147528537f839a2085d84" is_pre_election: true
I20260926 11:43:34.811956 22570 raft_consensus.cc:2454] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 520628b8ec3e40a6837bd876ebc09433 in term 0.
I20260926 11:43:34.811956 22841 raft_consensus.cc:2454] T 8f38c941dbd446a7867ba76c9458f9bd P 6afe305187804e80882f2700bdd14972 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 520628b8ec3e40a6837bd876ebc09433 in term 0.
I20260926 11:43:34.812394 22641 leader_election.cc:302] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 520628b8ec3e40a6837bd876ebc09433, 6afe305187804e80882f2700bdd14972; no voters:
I20260926 11:43:34.812647 22904 raft_consensus.cc:2781] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:43:34.812734 22904 raft_consensus.cc:482] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:43:34.812781 22904 raft_consensus.cc:3037] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:43:34.813789 22904 raft_consensus.cc:504] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:43:34.814143 22904 leader_election.cc:288] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433 [CANDIDATE]: Term 1 election: Requested vote from peers 6afe305187804e80882f2700bdd14972 (127.6.96.3:34785), 38b24333651147528537f839a2085d84 (127.6.96.1:44571)
I20260926 11:43:34.814541 22841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f38c941dbd446a7867ba76c9458f9bd" candidate_uuid: "520628b8ec3e40a6837bd876ebc09433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6afe305187804e80882f2700bdd14972"
I20260926 11:43:34.814574 22570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f38c941dbd446a7867ba76c9458f9bd" candidate_uuid: "520628b8ec3e40a6837bd876ebc09433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b24333651147528537f839a2085d84"
I20260926 11:43:34.814675 22570 raft_consensus.cc:3037] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:43:34.814674 22841 raft_consensus.cc:3037] T 8f38c941dbd446a7867ba76c9458f9bd P 6afe305187804e80882f2700bdd14972 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:43:34.815948 22570 raft_consensus.cc:2454] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 520628b8ec3e40a6837bd876ebc09433 in term 1.
I20260926 11:43:34.815948 22841 raft_consensus.cc:2454] T 8f38c941dbd446a7867ba76c9458f9bd P 6afe305187804e80882f2700bdd14972 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 520628b8ec3e40a6837bd876ebc09433 in term 1.
I20260926 11:43:34.816253 22641 leader_election.cc:302] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 520628b8ec3e40a6837bd876ebc09433, 6afe305187804e80882f2700bdd14972; no voters:
I20260926 11:43:34.816560 22904 raft_consensus.cc:2781] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:43:34.816757 22904 raft_consensus.cc:686] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433 [term 1 LEADER]: Becoming Leader. State: Replica: 520628b8ec3e40a6837bd876ebc09433, State: Running, Role: LEADER
I20260926 11:43:34.817122 22904 consensus_queue.cc:237] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:43:34.821144 22426 catalog_manager.cc:5495] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433 reported cstate change: term changed from 0 to 1, leader changed from <none> to 520628b8ec3e40a6837bd876ebc09433 (127.6.96.2). New cstate: current_term: 1 leader_uuid: "520628b8ec3e40a6837bd876ebc09433" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } health_report { overall_health: UNKNOWN } } }
I20260926 11:43:34.847229 22898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:34.843794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41278 (local address 127.6.96.2:44021)
0926 11:43:34.843935 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:34.843940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:34.843954 (+ 14us) server_negotiation.cc:407] Connection header received
0926 11:43:34.844040 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:34.844043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:34.844099 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:34.844192 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:43:34.844415 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.844925 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.845740 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:34.845967 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:34.846053 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:34.846373 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:43:34.846810 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:43:34.846956 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 11:43:34.847010 (+ 54us) server_negotiation.cc:299] Negotiation successful
0926 11:43:34.847072 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 11:43:34.853228 22841 raft_consensus.cc:1261] T 8f38c941dbd446a7867ba76c9458f9bd P 6afe305187804e80882f2700bdd14972 [term 1 FOLLOWER]: Refusing update from remote peer 520628b8ec3e40a6837bd876ebc09433: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:43:34.853268 22570 raft_consensus.cc:1261] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Refusing update from remote peer 520628b8ec3e40a6837bd876ebc09433: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:43:34.853873 22904 consensus_queue.cc:1035] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:43:34.854074 22906 consensus_queue.cc:1035] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:43:34.865104 22911 mvcc.cc:205] Tried to move back new op lower bound from 7333572668831797248 to 7333572668694155264. Current Snapshot: MvccSnapshot[applied={T|T < 7333572668831797248}]
I20260926 11:43:34.872694 22428 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:43:34.872870 22428 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:43:34.875229 22428 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9: Sending DeleteTablet for 3 replicas of tablet 8f38c941dbd446a7867ba76c9458f9bd
I20260926 11:43:34.876003 22550 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f38c941dbd446a7867ba76c9458f9bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:43:34 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:40888
I20260926 11:43:34.876075 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 22466
I20260926 11:43:34.876266 22821 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f38c941dbd446a7867ba76c9458f9bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:43:34 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:34526
I20260926 11:43:34.877535 22918 tablet_replica.cc:330] T 8f38c941dbd446a7867ba76c9458f9bd P 6afe305187804e80882f2700bdd14972: stopping tablet replica
I20260926 11:43:34.877625 22689 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f38c941dbd446a7867ba76c9458f9bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:43:34 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:41262
I20260926 11:43:34.877899 22918 raft_consensus.cc:2229] T 8f38c941dbd446a7867ba76c9458f9bd P 6afe305187804e80882f2700bdd14972 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:43:34.878134 22919 tablet_replica.cc:330] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433: stopping tablet replica
I20260926 11:43:34.878206 22918 raft_consensus.cc:2258] T 8f38c941dbd446a7867ba76c9458f9bd P 6afe305187804e80882f2700bdd14972 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:43:34.878468 22919 raft_consensus.cc:2229] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433 [term 1 LEADER]: Raft consensus shutting down.
I20260926 11:43:34.878875 22919 raft_consensus.cc:2258] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:43:34.879769 22918 ts_tablet_manager.cc:1905] T 8f38c941dbd446a7867ba76c9458f9bd P 6afe305187804e80882f2700bdd14972: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:43:34.880218 22919 ts_tablet_manager.cc:1905] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:43:34.882714 22919 ts_tablet_manager.cc:1918] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:43:34.882725 22918 ts_tablet_manager.cc:1918] T 8f38c941dbd446a7867ba76c9458f9bd P 6afe305187804e80882f2700bdd14972: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:43:34.882802 22919 log.cc:1196] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/wal/wals/8f38c941dbd446a7867ba76c9458f9bd
I20260926 11:43:34.882819 22918 log.cc:1196] T 8f38c941dbd446a7867ba76c9458f9bd P 6afe305187804e80882f2700bdd14972: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/8f38c941dbd446a7867ba76c9458f9bd
I20260926 11:43:34.883080 22919 ts_tablet_manager.cc:1939] T 8f38c941dbd446a7867ba76c9458f9bd P 520628b8ec3e40a6837bd876ebc09433: Deleting consensus metadata
I20260926 11:43:34.883206 22918 ts_tablet_manager.cc:1939] T 8f38c941dbd446a7867ba76c9458f9bd P 6afe305187804e80882f2700bdd14972: Deleting consensus metadata
W20260926 11:43:34.883801 22412 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.96.1:44571 (error 108)
I20260926 11:43:34.884476 22415 catalog_manager.cc:4841] TS 520628b8ec3e40a6837bd876ebc09433 (127.6.96.2:44021): tablet 8f38c941dbd446a7867ba76c9458f9bd (table test-table [id=80151774064948a9b67dbc8af1b9a225]) successfully deleted
I20260926 11:43:34.884508 22412 catalog_manager.cc:4841] TS 6afe305187804e80882f2700bdd14972 (127.6.96.3:34785): tablet 8f38c941dbd446a7867ba76c9458f9bd (table test-table [id=80151774064948a9b67dbc8af1b9a225]) successfully deleted
I20260926 11:43:34.884948 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 22620
I20260926 11:43:34.886158 22896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:34.885737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:44571 (local address 127.0.0.1:40902)
0926 11:43:34.886030 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:34.886088 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.1:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217}
W20260926 11:43:34.886309 22412 catalog_manager.cc:4584] TS 38b24333651147528537f839a2085d84 (127.6.96.1:44571): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8f38c941dbd446a7867ba76c9458f9bd: Network error: Client connection negotiation failed: client connection to 127.6.96.1:44571: connect: Connection refused (error 111)
I20260926 11:43:34.890458 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 22758
I20260926 11:43:34.895417 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 22395
I20260926 11:43:54.901490 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:45691
--webserver_interface=127.6.96.62
--webserver_port=36629
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:45271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:45691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:43:55.011334 22921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:43:55.011619 22921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:43:55.011662 22921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:43:55.015691 22921 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 11:43:55.015751 22921 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 11:43:55.015766 22921 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 11:43:55.015779 22921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:43:55.015802 22921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:43:55.015820 22921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:43:55.015834 22921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:43:55.015851 22921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:43:55.021245 22921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:45271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:45691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:45691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=36629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:43:55.022312 22921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:43:55.023375 22921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:43:55.032193 22927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:43:55.032353 22921 server_base.cc:1034] running on GCE node
W20260926 11:43:55.032202 22929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:43:55.032214 22926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:43:55.033125 22921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:43:55.034209 22921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:43:55.035365 22921 hybrid_clock.cc:648] HybridClock initialized: now 1790423035035348 us; error 31 us; skew 500 ppm
Sep 26 11:43:55 dist-test-slave-xxxx krb5kdc[21847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423035, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:43:55.046955 22921 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:43:55.047993 22921 webserver.cc:460] Webserver started at http://127.6.96.62:36629/ using document root <none> and password file <none>
I20260926 11:43:55.048529 22921 fs_manager.cc:359] Metadata directory not provided
I20260926 11:43:55.048595 22921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:43:55.052512 22921 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 11:43:55.054663 22936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:55.055959 22921 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:43:55.056066 22921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "6395ba7907354546ae2ae6772a73a5d9"
format_stamp: "Formatted at 2026-09-26 11:43:12 on dist-test-slave-xxxx"
server_key: "481ac7e305a6400a47feaff7f8c77813"
server_key_iv: "489fec93914a580bc666f848124e9627"
server_key_version: "encryptionkey@0"
I20260926 11:43:55.056631 22921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:43:55.079949 22921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:43:55.082759 22921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:43:55.083007 22921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:43:55.090024 22921 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:45691
I20260926 11:43:55.090307 22988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:45691 every 8 connection(s)
I20260926 11:43:55.091171 22921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:43:55.095465 22989 sys_catalog.cc:263] Verifying existing consensus state
I20260926 11:43:55.097078 22989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9: Bootstrap starting.
I20260926 11:43:55.098161 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 22921
I20260926 11:43:55.098639 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:44571
--local_ip_for_outbound_sockets=127.6.96.1
--tserver_master_addrs=127.6.96.62:45691
--webserver_port=46711
--webserver_interface=127.6.96.1
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:45271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 11:43:55.114847 22989 log.cc:824] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9: Log is configured to *not* fsync() on all Append() calls
I20260926 11:43:55.126603 22989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 11:43:55.126998 22989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9: Bootstrap complete.
I20260926 11:43:55.130811 22989 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6395ba7907354546ae2ae6772a73a5d9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45691 } }
I20260926 11:43:55.131248 22989 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6395ba7907354546ae2ae6772a73a5d9, State: Initialized, Role: FOLLOWER
I20260926 11:43:55.131744 22989 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6395ba7907354546ae2ae6772a73a5d9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45691 } }
I20260926 11:43:55.131872 22989 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:43:55.131939 22989 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:43:55.132037 22989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 2 FOLLOWER]: Advancing to term 3
I20260926 11:43:55.133967 22989 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6395ba7907354546ae2ae6772a73a5d9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45691 } }
I20260926 11:43:55.134294 22989 leader_election.cc:302] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6395ba7907354546ae2ae6772a73a5d9; no voters:
I20260926 11:43:55.134584 22989 leader_election.cc:288] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 11:43:55.134723 22993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 11:43:55.134984 22993 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [term 3 LEADER]: Becoming Leader. State: Replica: 6395ba7907354546ae2ae6772a73a5d9, State: Running, Role: LEADER
I20260926 11:43:55.135327 22993 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6395ba7907354546ae2ae6772a73a5d9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45691 } }
I20260926 11:43:55.135800 22989 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:43:55.136336 22993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6395ba7907354546ae2ae6772a73a5d9. Latest consensus state: current_term: 3 leader_uuid: "6395ba7907354546ae2ae6772a73a5d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6395ba7907354546ae2ae6772a73a5d9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45691 } } }
I20260926 11:43:55.136453 22993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [sys.catalog]: This master's current role is: LEADER
I20260926 11:43:55.136844 22998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:43:55.136868 22994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6395ba7907354546ae2ae6772a73a5d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6395ba7907354546ae2ae6772a73a5d9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45691 } } }
I20260926 11:43:55.137072 22994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9 [sys.catalog]: This master's current role is: LEADER
I20260926 11:43:55.141978 22998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:43:55.142757 22998 catalog_manager.cc:1259] Loaded cluster ID: 8eae91f9b435482a8e9d2f9d9dde6031
I20260926 11:43:55.142858 22998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:43:55.145359 22998 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:43:55.146095 22998 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9: Loaded TSK: 0
I20260926 11:43:55.146179 22998 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6395ba7907354546ae2ae6772a73a5d9: Loaded TSK: 1
F20260926 11:43:55.146919 22998 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f21b200f9ad google::LogMessage::Fail() at ??:0
@ 0x7f21b201288a google::LogMessage::SendToLog() at ??:0
@ 0x7f21b200f4de google::LogMessage::Flush() at ??:0
@ 0x7f21b2010a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f21b4cd318d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f21b4ccbf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f21bc9747bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f21bc94e99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f21bc94f178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f21bc990df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f21bc9af82b std::function<>::operator()() at ??:0
@ 0x7f21bc94eb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f21bc94ff3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f21bc94c579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f21bc990640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f21b42613d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f21b4261c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f21b42633f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f21b4252512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f21b58966db start_thread at ??:0
@ 0x7f21b0b0d61f clone at ??:0
W20260926 11:43:55.232610 22991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:43:55.232887 22991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:43:55.232929 22991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:43:55.237198 22991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:43:55.237268 22991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:43:55.237347 22991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:43:55.242748 22991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:45271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:44571
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=46711
--tserver_master_addrs=127.6.96.62:45691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:43:55.243881 22991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:43:55.244956 22991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:43:55.254462 23037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:43:55.254693 22991 server_base.cc:1034] running on GCE node
W20260926 11:43:55.254456 23036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:43:55.254987 23039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:43:55.255471 22991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:43:55.256304 22991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:43:55.257512 22991 hybrid_clock.cc:648] HybridClock initialized: now 1790423035257482 us; error 38 us; skew 500 ppm
Sep 26 11:43:55 dist-test-slave-xxxx krb5kdc[21847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423035, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:43:55.260810 22991 init.cc:377] Logged in from keytab as kudu/127.6.96.1@KRBTEST.COM (short username kudu)
I20260926 11:43:55.261757 22991 webserver.cc:460] Webserver started at http://127.6.96.1:46711/ using document root <none> and password file <none>
I20260926 11:43:55.262308 22991 fs_manager.cc:359] Metadata directory not provided
I20260926 11:43:55.262369 22991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:43:55.266686 22991 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:43:55.269069 23046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:55.270423 22991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 11:43:55.270533 22991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "38b24333651147528537f839a2085d84"
format_stamp: "Formatted at 2026-09-26 11:43:13 on dist-test-slave-xxxx"
server_key: "324a2e58deb6fe391d9743567b8da01a"
server_key_iv: "5ed8a67423014a80c6574189d1c82c49"
server_key_version: "encryptionkey@0"
I20260926 11:43:55.271142 22991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:43:55.280866 22991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:43:55.284242 22991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:43:55.284426 22991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:43:55.285028 22991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:43:55.286134 23053 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 11:43:55.291702 22991 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 11:43:55.291779 22991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260926 11:43:55.291859 22991 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 11:43:55.293552 22991 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 11:43:55.293601 22991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260926 11:43:55.293948 23053 tablet_bootstrap.cc:492] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84: Bootstrap starting.
I20260926 11:43:55.303368 22991 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:44571
I20260926 11:43:55.303653 23160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:44571 every 8 connection(s)
I20260926 11:43:55.304389 22991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:43:55.305352 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 22991
I20260926 11:43:55.305753 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:44021
--local_ip_for_outbound_sockets=127.6.96.2
--tserver_master_addrs=127.6.96.62:45691
--webserver_port=43043
--webserver_interface=127.6.96.2
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:45271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 11:43:55.315819 23053 log.cc:824] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84: Log is configured to *not* fsync() on all Append() calls
I20260926 11:43:55.325639 23053 tablet_bootstrap.cc:492] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 11:43:55.326074 23053 tablet_bootstrap.cc:492] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84: Bootstrap complete.
I20260926 11:43:55.327406 23053 ts_tablet_manager.cc:1397] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84: Time spent bootstrapping tablet: real 0.034s user 0.023s sys 0.010s
I20260926 11:43:55.330772 23053 raft_consensus.cc:348] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:43:55.331367 23053 raft_consensus.cc:729] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b24333651147528537f839a2085d84, State: Initialized, Role: FOLLOWER
I20260926 11:43:55.331893 23053 consensus_queue.cc:260] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:43:55.332957 23053 ts_tablet_manager.cc:1428] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
W20260926 11:43:55.378970 23164 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:43:55.307562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.1:60117)
0926 11:43:55.308886 (+ 1324us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:55.308942 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 11:43:55.309953 (+ 1011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:43:55.377950 (+ 67997us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1086,"thread_start_us":147,"threads_started":1}
W20260926 11:43:55.379575 23161 heartbeater.cc:644] Failed to heartbeat to 127.6.96.62:45691 (0 consecutive failures): Network error: Failed to ping master at 127.6.96.62:45691: Client connection negotiation failed: client connection to 127.6.96.62:45691: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 11:43:55.380369 23164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:55.380084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.1:57473)
0926 11:43:55.380219 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:55.380296 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260926 11:43:55.381104 23164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:55.380888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.1:37603)
0926 11:43:55.380994 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:55.381044 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260926 11:43:55.381302 23161 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 11:43:55.420617 23163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:43:55.420913 23163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:43:55.420976 23163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:43:55.425257 23163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:43:55.425330 23163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:43:55.425411 23163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:43:55.430976 23163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:45271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:44021
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=43043
--tserver_master_addrs=127.6.96.62:45691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:43:55.432143 23163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:43:55.433246 23163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:43:55.443542 23171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:43:55.443562 23172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:43:55.443707 23163 server_base.cc:1034] running on GCE node
W20260926 11:43:55.443548 23174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:43:55.444274 23163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:43:55.444967 23163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:43:55.446149 23163 hybrid_clock.cc:648] HybridClock initialized: now 1790423035446121 us; error 40 us; skew 500 ppm
Sep 26 11:43:55 dist-test-slave-xxxx krb5kdc[21847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423035, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:43:55.449082 23163 init.cc:377] Logged in from keytab as kudu/127.6.96.2@KRBTEST.COM (short username kudu)
I20260926 11:43:55.450007 23163 webserver.cc:460] Webserver started at http://127.6.96.2:43043/ using document root <none> and password file <none>
I20260926 11:43:55.450570 23163 fs_manager.cc:359] Metadata directory not provided
I20260926 11:43:55.450659 23163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:43:55.454632 23163 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:43:55.456882 23181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:55.458150 23163 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:43:55.458254 23163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "520628b8ec3e40a6837bd876ebc09433"
format_stamp: "Formatted at 2026-09-26 11:43:13 on dist-test-slave-xxxx"
server_key: "ba1e388515aa42c7838d86eb783e303e"
server_key_iv: "451a289daf1c4f746f90ce8b7d821cdb"
server_key_version: "encryptionkey@0"
I20260926 11:43:55.458858 23163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:43:55.479795 23163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:43:55.482975 23163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:43:55.483155 23163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:43:55.483754 23163 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:43:55.484592 23163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:43:55.484637 23163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:55.484679 23163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:43:55.484709 23163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:55.493151 23163 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:44021
I20260926 11:43:55.493180 23294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:44021 every 8 connection(s)
I20260926 11:43:55.494166 23163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:43:55.497797 23297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:55.496285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.2:59121)
0926 11:43:55.496949 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:55.497070 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":112,"threads_started":1}
W20260926 11:43:55.498253 23295 heartbeater.cc:644] Failed to heartbeat to 127.6.96.62:45691 (0 consecutive failures): Network error: Failed to ping master at 127.6.96.62:45691: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
I20260926 11:43:55.498883 23297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:55.498644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.2:45145)
0926 11:43:55.498751 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:55.498800 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260926 11:43:55.499439 23297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:55.499250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.2:51859)
0926 11:43:55.499346 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:55.499389 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260926 11:43:55.499585 23295 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 11:43:55.503400 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 23163
I20260926 11:43:55.503822 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:34785
--local_ip_for_outbound_sockets=127.6.96.3
--tserver_master_addrs=127.6.96.62:45691
--webserver_port=35027
--webserver_interface=127.6.96.3
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:45271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:43:55.615700 23298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:43:55.615993 23298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:43:55.616037 23298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:43:55.620438 23298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:43:55.620505 23298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:43:55.620563 23298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:43:55.626157 23298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:45271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:34785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=35027
--tserver_master_addrs=127.6.96.62:45691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:43:55.627285 23298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:43:55.628305 23298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 11:43:55.638126 23298 server_base.cc:1034] running on GCE node
W20260926 11:43:55.638162 23303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:43:55.638150 23304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:43:55.638355 23306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:43:55.638890 23298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:43:55.639554 23298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:43:55.640743 23298 hybrid_clock.cc:648] HybridClock initialized: now 1790423035640705 us; error 57 us; skew 500 ppm
Sep 26 11:43:55 dist-test-slave-xxxx krb5kdc[21847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423035, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:43:55.643464 23298 init.cc:377] Logged in from keytab as kudu/127.6.96.3@KRBTEST.COM (short username kudu)
I20260926 11:43:55.644330 23298 webserver.cc:460] Webserver started at http://127.6.96.3:35027/ using document root <none> and password file <none>
I20260926 11:43:55.644877 23298 fs_manager.cc:359] Metadata directory not provided
I20260926 11:43:55.644944 23298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:43:55.648799 23298 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 11:43:55.650796 23313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:55.652064 23298 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:43:55.652189 23298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "6afe305187804e80882f2700bdd14972"
format_stamp: "Formatted at 2026-09-26 11:43:13 on dist-test-slave-xxxx"
server_key: "29f55bcb297d9fe96efe4145e68c4fa0"
server_key_iv: "4885cf12e371264cab2599288556efd1"
server_key_version: "encryptionkey@0"
I20260926 11:43:55.652721 23298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:43:55.665503 23298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:43:55.668548 23298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:43:55.668720 23298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:43:55.669297 23298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:43:55.670114 23298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:43:55.670174 23298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:55.670236 23298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:43:55.670259 23298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:43:55.679104 23298 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:34785
I20260926 11:43:55.679132 23426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:34785 every 8 connection(s)
I20260926 11:43:55.680119 23298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:43:55.683965 23429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:55.682292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.3:34337)
0926 11:43:55.683092 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:55.683216 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":123,"threads_started":1}
W20260926 11:43:55.684429 23427 heartbeater.cc:644] Failed to heartbeat to 127.6.96.62:45691 (0 consecutive failures): Network error: Failed to ping master at 127.6.96.62:45691: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
I20260926 11:43:55.685096 23429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:55.684854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.3:52885)
0926 11:43:55.684976 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:55.685034 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260926 11:43:55.685828 23429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:55.685578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.3:57091)
0926 11:43:55.685701 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:55.685749 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260926 11:43:55.686025 23427 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 11:43:55.690382 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 23298
I20260926 11:43:56.382437 23431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:56.381969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.1:51045)
0926 11:43:56.382248 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:56.382312 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":69,"threads_started":1}
I20260926 11:43:56.500548 23432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:56.500144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.2:41991)
0926 11:43:56.500391 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:56.500449 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":66,"threads_started":1}
I20260926 11:43:56.687052 23433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:56.686587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.3:49699)
0926 11:43:56.686891 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:56.686952 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":118,"threads_started":1}
I20260926 11:43:57.106036 23434 raft_consensus.cc:482] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:43:57.106233 23434 raft_consensus.cc:504] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:43:57.107362 23434 leader_election.cc:288] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6afe305187804e80882f2700bdd14972 (127.6.96.3:34785), 520628b8ec3e40a6837bd876ebc09433 (127.6.96.2:44021)
Sep 26 11:43:57 dist-test-slave-xxxx krb5kdc[21847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423035, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.3@KRBTEST.COM
Sep 26 11:43:57 dist-test-slave-xxxx krb5kdc[21847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423035, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.2@KRBTEST.COM
I20260926 11:43:57.120851 23438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:57.107828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:55523 (local address 127.6.96.3:34785)
0926 11:43:57.108416 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:57.108439 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:57.108479 (+ 40us) server_negotiation.cc:407] Connection header received
0926 11:43:57.109308 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:57.109314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:57.109613 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:57.109866 (+ 253us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:43:57.111017 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:57.112018 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:57.113152 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:57.113618 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:57.113764 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:57.116356 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:43:57.116363 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:43:57.116381 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:43:57.116415 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:43:57.118133 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:57.118703 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:57.118709 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:57.118714 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:57.118887 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:57.119234 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:57.119238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:57.119239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:57.119783 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:43:57.119905 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 11:43:57.120424 (+ 519us) server_negotiation.cc:299] Negotiation successful
0926 11:43:57.120667 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":92,"threads_started":1}
I20260926 11:43:57.120852 23436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:57.107663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:44021 (local address 127.6.96.1:52671)
0926 11:43:57.108029 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:57.108044 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:43:57.108164 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:43:57.109816 (+ 1652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:43:57.109823 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:43:57.110393 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:43:57.110863 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:57.110881 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:57.112140 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:57.112143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:57.113032 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:57.113042 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:57.113742 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:57.113746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:57.113909 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:57.114521 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:43:57.114537 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:43:57.116435 (+ 1898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:43:57.118381 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:43:57.118383 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:43:57.118385 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:43:57.118607 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:43:57.118975 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:43:57.118978 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:43:57.118979 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:43:57.119104 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:43:57.119933 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:43:57.119936 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:43:57.120135 (+ 199us) client_negotiation.cc:769] Sending connection context
0926 11:43:57.120303 (+ 168us) client_negotiation.cc:240] Negotiation successful
0926 11:43:57.120645 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":158,"spinlock_wait_cycles":27520,"thread_start_us":57,"threads_started":1}
I20260926 11:43:57.120873 23437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:57.107853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:52671 (local address 127.6.96.2:44021)
0926 11:43:57.108286 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0926 11:43:57.108303 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 11:43:57.108355 (+ 52us) server_negotiation.cc:407] Connection header received
0926 11:43:57.109101 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:43:57.109108 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:43:57.109453 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:43:57.109749 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:43:57.111019 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:57.111999 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:57.113161 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:43:57.113621 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:43:57.113763 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:57.116628 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:43:57.116633 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:43:57.116644 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:43:57.116671 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:43:57.118274 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:57.118713 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:57.118717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:57.118722 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:57.118864 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:43:57.119200 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:43:57.119204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:43:57.119206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:43:57.119831 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:43:57.119947 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 11:43:57.120423 (+ 476us) server_negotiation.cc:299] Negotiation successful
0926 11:43:57.120665 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":58,"threads_started":1}
I20260926 11:43:57.120883 23435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:57.107697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:34785 (local address 127.6.96.1:55523)
0926 11:43:57.108255 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:57.108268 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:43:57.108615 (+ 347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:43:57.109823 (+ 1208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:43:57.109826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:43:57.110388 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:43:57.110864 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:57.110882 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:57.112146 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:57.112149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:57.113029 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:43:57.113038 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:43:57.113732 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:43:57.113736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:43:57.113910 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:43:57.114571 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:43:57.114583 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:43:57.116123 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:43:57.118261 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:43:57.118267 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:43:57.118281 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:43:57.118597 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:43:57.118994 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:43:57.118997 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:43:57.118998 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:43:57.119122 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:43:57.119897 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:43:57.119903 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:43:57.120132 (+ 229us) client_negotiation.cc:769] Sending connection context
0926 11:43:57.120304 (+ 172us) client_negotiation.cc:240] Negotiation successful
0926 11:43:57.120730 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":189,"spinlock_wait_cycles":16896,"thread_start_us":86,"threads_started":1}
W20260926 11:43:57.123478 23047 leader_election.cc:341] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6afe305187804e80882f2700bdd14972 (127.6.96.3:34785): Not found: Tablet not found: 8f38c941dbd446a7867ba76c9458f9bd
W20260926 11:43:57.123734 23050 leader_election.cc:341] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 520628b8ec3e40a6837bd876ebc09433 (127.6.96.2:44021): Not found: Tablet not found: 8f38c941dbd446a7867ba76c9458f9bd
I20260926 11:43:57.123842 23050 leader_election.cc:302] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b24333651147528537f839a2085d84; no voters: 520628b8ec3e40a6837bd876ebc09433, 6afe305187804e80882f2700bdd14972
I20260926 11:43:57.124083 23434 raft_consensus.cc:2726] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 11:43:57.383569 23436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:57.383295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.1:46253)
0926 11:43:57.383433 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:57.383488 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260926 11:43:57.501803 23439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:57.501366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.2:59463)
0926 11:43:57.501637 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:57.501697 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":71,"threads_started":1}
I20260926 11:43:57.688387 23440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:57.687909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.3:43623)
0926 11:43:57.688208 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:57.688275 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":80,"threads_started":1}
I20260926 11:43:58.384815 23441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:58.384379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.1:44339)
0926 11:43:58.384632 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:58.384695 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":77,"threads_started":1}
I20260926 11:43:58.503042 23442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:58.502581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.2:40067)
0926 11:43:58.502876 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:58.502938 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":84,"threads_started":1}
I20260926 11:43:58.689723 23443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:58.689245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.3:54705)
0926 11:43:58.689562 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:58.689627 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":76,"threads_started":1}
I20260926 11:43:59.137583 23444 raft_consensus.cc:482] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:43:59.137722 23444 raft_consensus.cc:504] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:43:59.138134 23444 leader_election.cc:288] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6afe305187804e80882f2700bdd14972 (127.6.96.3:34785), 520628b8ec3e40a6837bd876ebc09433 (127.6.96.2:44021)
W20260926 11:43:59.138953 23050 leader_election.cc:341] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 520628b8ec3e40a6837bd876ebc09433 (127.6.96.2:44021): Not found: Tablet not found: 8f38c941dbd446a7867ba76c9458f9bd
W20260926 11:43:59.139073 23047 leader_election.cc:341] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6afe305187804e80882f2700bdd14972 (127.6.96.3:34785): Not found: Tablet not found: 8f38c941dbd446a7867ba76c9458f9bd
I20260926 11:43:59.139158 23047 leader_election.cc:302] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b24333651147528537f839a2085d84; no voters: 520628b8ec3e40a6837bd876ebc09433, 6afe305187804e80882f2700bdd14972
I20260926 11:43:59.139318 23444 raft_consensus.cc:2726] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 11:43:59.386040 23445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:59.385576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.1:52227)
0926 11:43:59.385880 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:59.385942 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":98,"threads_started":1}
I20260926 11:43:59.504160 23446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:59.503751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.2:58711)
0926 11:43:59.503996 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:59.504052 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":89,"threads_started":1}
I20260926 11:43:59.690996 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:43:59.690515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.3:43545)
0926 11:43:59.690797 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0926 11:43:59.690901 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":93,"threads_started":1}
I20260926 11:44:00.387408 23448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:00.386959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.1:49931)
0926 11:44:00.387248 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:00.387307 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":76,"threads_started":1}
I20260926 11:44:00.505383 23449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:00.504961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.2:36589)
0926 11:44:00.505212 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:00.505273 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":71,"threads_started":1}
I20260926 11:44:00.692129 23450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:00.691717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.3:38985)
0926 11:44:00.691980 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:00.692036 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":91,"threads_started":1}
I20260926 11:44:01.388656 23451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:01.388206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.1:40405)
0926 11:44:01.388495 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:01.388558 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":75,"threads_started":1}
I20260926 11:44:01.506542 23452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:01.506131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.2:53781)
0926 11:44:01.506382 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:01.506443 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":73,"threads_started":1}
I20260926 11:44:01.693400 23453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:01.692976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.3:49741)
0926 11:44:01.693238 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:01.693303 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":72,"threads_started":1}
I20260926 11:44:02.068053 23454 raft_consensus.cc:482] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:44:02.068183 23454 raft_consensus.cc:504] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:44:02.068578 23454 leader_election.cc:288] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6afe305187804e80882f2700bdd14972 (127.6.96.3:34785), 520628b8ec3e40a6837bd876ebc09433 (127.6.96.2:44021)
W20260926 11:44:02.069276 23050 leader_election.cc:341] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 520628b8ec3e40a6837bd876ebc09433 (127.6.96.2:44021): Not found: Tablet not found: 8f38c941dbd446a7867ba76c9458f9bd
W20260926 11:44:02.069358 23047 leader_election.cc:341] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6afe305187804e80882f2700bdd14972 (127.6.96.3:34785): Not found: Tablet not found: 8f38c941dbd446a7867ba76c9458f9bd
I20260926 11:44:02.069453 23047 leader_election.cc:302] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b24333651147528537f839a2085d84; no voters: 520628b8ec3e40a6837bd876ebc09433, 6afe305187804e80882f2700bdd14972
I20260926 11:44:02.069594 23454 raft_consensus.cc:2726] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 11:44:02.389925 23455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:02.389464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.1:54285)
0926 11:44:02.389763 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:02.389826 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":93,"threads_started":1}
I20260926 11:44:02.507858 23456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:02.507369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.2:34165)
0926 11:44:02.507693 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:02.507759 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":74,"threads_started":1}
I20260926 11:44:02.694731 23457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:02.694251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.3:42211)
0926 11:44:02.694574 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:02.694638 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":75,"threads_started":1}
I20260926 11:44:03.391238 23458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:03.390758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.1:53209)
0926 11:44:03.391068 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:03.391137 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":112,"threads_started":1}
I20260926 11:44:03.509092 23459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:03.508630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.2:49613)
0926 11:44:03.508929 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:03.508994 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":78,"threads_started":1}
I20260926 11:44:03.695899 23460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:03.695476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.3:58751)
0926 11:44:03.695739 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:03.695805 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":90,"threads_started":1}
I20260926 11:44:04.392400 23461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:04.391973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.1:35979)
0926 11:44:04.392237 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:04.392301 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":79,"threads_started":1}
I20260926 11:44:04.510250 23462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:04.509835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.2:53737)
0926 11:44:04.510081 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:04.510140 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":72,"threads_started":1}
I20260926 11:44:04.697151 23463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:04.696711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.3:43933)
0926 11:44:04.696996 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:04.697055 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":91,"threads_started":1}
I20260926 11:44:05.393615 23464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:05.393192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.1:47163)
0926 11:44:05.393466 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:05.393520 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":90,"threads_started":1}
I20260926 11:44:05.511507 23465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:05.511046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.2:51655)
0926 11:44:05.511339 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:05.511412 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":86,"threads_started":1}
I20260926 11:44:05.698513 23466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:05.697997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.3:49767)
0926 11:44:05.698336 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:05.698412 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":93,"threads_started":1}
I20260926 11:44:06.318058 23467 raft_consensus.cc:482] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:44:06.318197 23467 raft_consensus.cc:504] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:44:06.318624 23467 leader_election.cc:288] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6afe305187804e80882f2700bdd14972 (127.6.96.3:34785), 520628b8ec3e40a6837bd876ebc09433 (127.6.96.2:44021)
W20260926 11:44:06.319489 23047 leader_election.cc:341] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6afe305187804e80882f2700bdd14972 (127.6.96.3:34785): Not found: Tablet not found: 8f38c941dbd446a7867ba76c9458f9bd
W20260926 11:44:06.319612 23050 leader_election.cc:341] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 520628b8ec3e40a6837bd876ebc09433 (127.6.96.2:44021): Not found: Tablet not found: 8f38c941dbd446a7867ba76c9458f9bd
I20260926 11:44:06.319674 23050 leader_election.cc:302] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b24333651147528537f839a2085d84; no voters: 520628b8ec3e40a6837bd876ebc09433, 6afe305187804e80882f2700bdd14972
I20260926 11:44:06.319824 23467 raft_consensus.cc:2726] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 11:44:06.394821 23468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:06.394367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.1:37583)
0926 11:44:06.394656 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:06.394717 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":90,"threads_started":1}
I20260926 11:44:06.512660 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:06.512211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.2:43373)
0926 11:44:06.512500 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:06.512563 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":88,"threads_started":1}
I20260926 11:44:06.699792 23470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:06.699295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.3:50343)
0926 11:44:06.699606 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:06.699673 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":95,"threads_started":1}
I20260926 11:44:07.396008 23471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:07.395565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.1:59687)
0926 11:44:07.395851 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:07.395908 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":73,"threads_started":1}
I20260926 11:44:07.514009 23472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:07.513537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.2:39313)
0926 11:44:07.513842 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:07.513909 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":90,"threads_started":1}
I20260926 11:44:07.701083 23473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:07.700638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.3:44697)
0926 11:44:07.700923 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:07.700990 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":77,"threads_started":1}
I20260926 11:44:08.172585 23474 raft_consensus.cc:482] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:44:08.172726 23474 raft_consensus.cc:504] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afe305187804e80882f2700bdd14972" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 34785 } } peers { permanent_uuid: "520628b8ec3e40a6837bd876ebc09433" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 44021 } } peers { permanent_uuid: "38b24333651147528537f839a2085d84" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 44571 } }
I20260926 11:44:08.173138 23474 leader_election.cc:288] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6afe305187804e80882f2700bdd14972 (127.6.96.3:34785), 520628b8ec3e40a6837bd876ebc09433 (127.6.96.2:44021)
W20260926 11:44:08.173936 23050 leader_election.cc:341] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 520628b8ec3e40a6837bd876ebc09433 (127.6.96.2:44021): Not found: Tablet not found: 8f38c941dbd446a7867ba76c9458f9bd
W20260926 11:44:08.174073 23047 leader_election.cc:341] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6afe305187804e80882f2700bdd14972 (127.6.96.3:34785): Not found: Tablet not found: 8f38c941dbd446a7867ba76c9458f9bd
I20260926 11:44:08.174136 23047 leader_election.cc:302] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b24333651147528537f839a2085d84; no voters: 520628b8ec3e40a6837bd876ebc09433, 6afe305187804e80882f2700bdd14972
I20260926 11:44:08.174299 23474 raft_consensus.cc:2726] T 8f38c941dbd446a7867ba76c9458f9bd P 38b24333651147528537f839a2085d84 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 11:44:08.397310 23475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:08.396798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.1:39943)
0926 11:44:08.397120 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:08.397188 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":121,"threads_started":1}
I20260926 11:44:08.515128 23476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:08.514674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.2:53709)
0926 11:44:08.514969 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:08.515029 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":79,"threads_started":1}
I20260926 11:44:08.702256 23477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:08.701799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.3:33727)
0926 11:44:08.702087 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:08.702158 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":70,"threads_started":1}
I20260926 11:44:09.398631 23478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:09.398167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.1:50939)
0926 11:44:09.398459 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:09.398527 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":98,"threads_started":1}
I20260926 11:44:09.516367 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:09.515917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.2:56539)
0926 11:44:09.516202 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:09.516267 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":75,"threads_started":1}
I20260926 11:44:09.703624 23480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:09.703180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.3:46939)
0926 11:44:09.703460 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:09.703527 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":84,"threads_started":1}
I20260926 11:44:10.400045 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:10.399663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.1:47043)
0926 11:44:10.399895 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:10.399949 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":73,"threads_started":1}
I20260926 11:44:10.517799 23482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:10.517335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:45691 (local address 127.6.96.2:48963)
0926 11:44:10.517609 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:10.517676 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.62:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":71,"threads_started":1}
I20260926 11:44:10.692694 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 22991
I20260926 11:44:10.697685 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 23163
I20260926 11:44:10.702346 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 23298
2026-09-26T11:44:10Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58967 ms)
[----------] 30 tests from SecurityITest (233195 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:44:10 dist-test-slave-xxxx krb5kdc[23485](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:44:10 dist-test-slave-xxxx krb5kdc[23485](info): set up 2 sockets
Sep 26 11:44:10 dist-test-slave-xxxx krb5kdc[23485](info): commencing operation
krb5kdc: starting...
W20260926 11:44:11.662472 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:44:11 dist-test-slave-xxxx krb5kdc[23485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:44:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:44:11.777773 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:40223
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:42645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:40223
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:11.884018 23498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:11.884294 23498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:11.884349 23498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:11.888355 23498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:11.888424 23498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:11.888451 23498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:11.888470 23498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:11.888492 23498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:11.893760 23498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:42645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:40223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:40223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:11.894817 23498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:11.895848 23498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:11.904028 23503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:11.904165 23506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:11.904165 23504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:11.904714 23498 server_base.cc:1034] running on GCE node
I20260926 11:44:11.905269 23498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:11.906374 23498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:11.907519 23498 hybrid_clock.cc:648] HybridClock initialized: now 1790423051907493 us; error 43 us; skew 500 ppm
Sep 26 11:44:11 dist-test-slave-xxxx krb5kdc[23485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423051, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:11.910054 23498 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:44:11.910975 23498 webserver.cc:460] Webserver started at http://127.6.96.62:34907/ using document root <none> and password file <none>
I20260926 11:44:11.911516 23498 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:11.911583 23498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:11.911792 23498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:11.913861 23498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "3b20543e9ea64587984bdb98f72e9f6f"
format_stamp: "Formatted at 2026-09-26 11:44:11 on dist-test-slave-xxxx"
server_key: "919b7c2f67275cd6d7e138aeca0b0785"
server_key_iv: "8fc6b8cdf9ddadb5b620bfda8dcb92b9"
server_key_version: "encryptionkey@0"
I20260926 11:44:11.914397 23498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "3b20543e9ea64587984bdb98f72e9f6f"
format_stamp: "Formatted at 2026-09-26 11:44:11 on dist-test-slave-xxxx"
server_key: "919b7c2f67275cd6d7e138aeca0b0785"
server_key_iv: "8fc6b8cdf9ddadb5b620bfda8dcb92b9"
server_key_version: "encryptionkey@0"
I20260926 11:44:11.918030 23498 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 11:44:11.920430 23513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:11.921627 23498 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 11:44:11.921757 23498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "3b20543e9ea64587984bdb98f72e9f6f"
format_stamp: "Formatted at 2026-09-26 11:44:11 on dist-test-slave-xxxx"
server_key: "919b7c2f67275cd6d7e138aeca0b0785"
server_key_iv: "8fc6b8cdf9ddadb5b620bfda8dcb92b9"
server_key_version: "encryptionkey@0"
I20260926 11:44:11.921856 23498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:11.942425 23498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:11.945590 23498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:11.945772 23498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:11.952333 23498 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:40223
I20260926 11:44:11.952360 23565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:40223 every 8 connection(s)
I20260926 11:44:11.953306 23498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:11.956054 23566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:11.961491 23566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b20543e9ea64587984bdb98f72e9f6f: Bootstrap starting.
I20260926 11:44:11.963385 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 23498
I20260926 11:44:11.963485 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:11.963740 6528 external_mini_cluster.cc:1428] Setting key bbb156054d0d76fcfdcb1284e0212daf
I20260926 11:44:11.963995 23566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b20543e9ea64587984bdb98f72e9f6f: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:11.964807 23566 log.cc:824] T 00000000000000000000000000000000 P 3b20543e9ea64587984bdb98f72e9f6f: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:11.966681 23566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b20543e9ea64587984bdb98f72e9f6f: No bootstrap required, opened a new log
I20260926 11:44:11.970070 23566 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b20543e9ea64587984bdb98f72e9f6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20543e9ea64587984bdb98f72e9f6f" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40223 } }
I20260926 11:44:11.970297 23566 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b20543e9ea64587984bdb98f72e9f6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:11.970353 23566 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b20543e9ea64587984bdb98f72e9f6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b20543e9ea64587984bdb98f72e9f6f, State: Initialized, Role: FOLLOWER
I20260926 11:44:11.970789 23566 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b20543e9ea64587984bdb98f72e9f6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20543e9ea64587984bdb98f72e9f6f" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40223 } }
I20260926 11:44:11.970956 23566 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b20543e9ea64587984bdb98f72e9f6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:44:11.971024 23566 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b20543e9ea64587984bdb98f72e9f6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:44:11.971123 23566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b20543e9ea64587984bdb98f72e9f6f [term 0 FOLLOWER]: Advancing to term 1
Sep 26 11:44:11 dist-test-slave-xxxx krb5kdc[23485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:44:11.972201 23566 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b20543e9ea64587984bdb98f72e9f6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20543e9ea64587984bdb98f72e9f6f" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40223 } }
I20260926 11:44:11.972553 23566 leader_election.cc:302] T 00000000000000000000000000000000 P 3b20543e9ea64587984bdb98f72e9f6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b20543e9ea64587984bdb98f72e9f6f; no voters:
I20260926 11:44:11.972887 23566 leader_election.cc:288] T 00000000000000000000000000000000 P 3b20543e9ea64587984bdb98f72e9f6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:44:11.972959 23571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b20543e9ea64587984bdb98f72e9f6f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:11.973138 23571 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b20543e9ea64587984bdb98f72e9f6f [term 1 LEADER]: Becoming Leader. State: Replica: 3b20543e9ea64587984bdb98f72e9f6f, State: Running, Role: LEADER
I20260926 11:44:11.973433 23571 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b20543e9ea64587984bdb98f72e9f6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20543e9ea64587984bdb98f72e9f6f" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40223 } }
I20260926 11:44:11.974207 23566 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b20543e9ea64587984bdb98f72e9f6f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:11.975152 23573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b20543e9ea64587984bdb98f72e9f6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b20543e9ea64587984bdb98f72e9f6f. Latest consensus state: current_term: 1 leader_uuid: "3b20543e9ea64587984bdb98f72e9f6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20543e9ea64587984bdb98f72e9f6f" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40223 } } }
I20260926 11:44:11.975608 23573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b20543e9ea64587984bdb98f72e9f6f [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:11.975502 23572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b20543e9ea64587984bdb98f72e9f6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b20543e9ea64587984bdb98f72e9f6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b20543e9ea64587984bdb98f72e9f6f" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40223 } } }
I20260926 11:44:11.975996 23572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b20543e9ea64587984bdb98f72e9f6f [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:11.976508 23582 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:11.977998 23569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:11.964820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49112 (local address 127.6.96.62:40223)
0926 11:44:11.965299 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:11.965320 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:11.965364 (+ 44us) server_negotiation.cc:407] Connection header received
0926 11:44:11.966063 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:11.966081 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:11.966410 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:11.966715 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:11.967554 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:11.968530 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:11.969392 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:11.969872 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:11.970033 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:11.972747 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:11.972752 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:11.972767 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:11.972796 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:11.974545 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:11.975103 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:11.975110 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:11.975116 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:11.975271 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:11.975591 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:11.975595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:11.975597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:11.976173 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:11.976371 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:11.977042 (+ 671us) server_negotiation.cc:299] Negotiation successful
0926 11:44:11.977274 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":150,"threads_started":1}
I20260926 11:44:11.979835 23582 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:11.985566 23582 catalog_manager.cc:1347] Generated new cluster ID: 7eee2ef1a2694f298bc5d5308a393240
I20260926 11:44:11.985625 23582 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:12.010479 23582 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:12.011821 23582 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:12.019383 23582 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b20543e9ea64587984bdb98f72e9f6f: Generated new TSK 0
I20260926 11:44:12.020216 23582 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.96.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
I20260926 11:44:12.049610 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:40223
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:42645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:12.160467 23592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:12.160751 23592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:12.160815 23592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:12.164889 23592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:12.164947 23592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:12.165028 23592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:44:12.170271 23592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:42645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:40223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:12.171376 23592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:12.172384 23592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:12.181521 23600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:12.181562 23598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:12.181516 23597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:12.181771 23592 server_base.cc:1034] running on GCE node
I20260926 11:44:12.182211 23592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:12.182870 23592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:12.184059 23592 hybrid_clock.cc:648] HybridClock initialized: now 1790423052184022 us; error 30 us; skew 500 ppm
Sep 26 11:44:12 dist-test-slave-xxxx krb5kdc[23485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423052, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:12.186786 23592 init.cc:377] Logged in from keytab as kudu/127.6.96.1@KRBTEST.COM (short username kudu)
I20260926 11:44:12.187708 23592 webserver.cc:460] Webserver started at http://127.6.96.1:33405/ using document root <none> and password file <none>
I20260926 11:44:12.188252 23592 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:12.188351 23592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:12.188555 23592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:12.190583 23592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "49ba1d8b81c94e6d91175975d65db5ce"
format_stamp: "Formatted at 2026-09-26 11:44:12 on dist-test-slave-xxxx"
server_key: "b88e531514181b5d5db979c07e20995b"
server_key_iv: "521ff6f670af12c8d3d6d9039755f2da"
server_key_version: "encryptionkey@0"
I20260926 11:44:12.191174 23592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "49ba1d8b81c94e6d91175975d65db5ce"
format_stamp: "Formatted at 2026-09-26 11:44:12 on dist-test-slave-xxxx"
server_key: "b88e531514181b5d5db979c07e20995b"
server_key_iv: "521ff6f670af12c8d3d6d9039755f2da"
server_key_version: "encryptionkey@0"
I20260926 11:44:12.194746 23592 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:44:12.197419 23607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:12.198873 23592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:44:12.198987 23592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "49ba1d8b81c94e6d91175975d65db5ce"
format_stamp: "Formatted at 2026-09-26 11:44:12 on dist-test-slave-xxxx"
server_key: "b88e531514181b5d5db979c07e20995b"
server_key_iv: "521ff6f670af12c8d3d6d9039755f2da"
server_key_version: "encryptionkey@0"
I20260926 11:44:12.199098 23592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:12.211683 23592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:12.214401 23592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:12.214579 23592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:12.215209 23592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:44:12.216042 23592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:44:12.216104 23592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:12.216158 23592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:44:12.216184 23592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:12.225064 23592 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:45135
I20260926 11:44:12.225061 23720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:45135 every 8 connection(s)
I20260926 11:44:12.226122 23592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
Sep 26 11:44:12 dist-test-slave-xxxx krb5kdc[23485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423052, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:44:12.235946 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 23592
I20260926 11:44:12.236052 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:44:12.236318 6528 external_mini_cluster.cc:1428] Setting key 92a4793f3e323177779353ea540ab371
I20260926 11:44:12.240139 23569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:12.227987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:56611 (local address 127.6.96.62:40223)
0926 11:44:12.228152 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:12.228156 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:12.228999 (+ 843us) server_negotiation.cc:407] Connection header received
0926 11:44:12.230332 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:12.230336 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:12.230393 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:12.230468 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:12.231717 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.232324 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.233476 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.233721 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.233809 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:12.236264 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:12.236267 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:12.236270 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:12.236295 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:12.237865 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:12.238489 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:12.238492 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:12.238493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:12.238578 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:12.238945 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:12.238948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:12.238949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:12.239137 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:12.239211 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:12.239823 (+ 612us) server_negotiation.cc:299] Negotiation successful
0926 11:44:12.239974 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 11:44:12.240888 23723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:12.228362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:40223 (local address 127.6.96.1:56611)
0926 11:44:12.228838 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:12.228883 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:12.230100 (+ 1217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:12.230660 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:12.230667 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:12.231072 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:12.231561 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:12.231575 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.232450 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.232453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:12.233357 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:12.233367 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.233866 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.233870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:12.234028 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:12.234551 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:12.234565 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:12.236094 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:12.237980 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:12.237986 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:12.238004 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:12.238376 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:12.238659 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:12.238662 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:12.238664 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:12.238863 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:12.239232 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:12.239239 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:12.239570 (+ 331us) client_negotiation.cc:769] Sending connection context
0926 11:44:12.239730 (+ 160us) client_negotiation.cc:240] Negotiation successful
0926 11:44:12.240001 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":108,"threads_started":1}
I20260926 11:44:12.242038 23721 heartbeater.cc:347] Connected to a master server at 127.6.96.62:40223
I20260926 11:44:12.242260 23721 heartbeater.cc:464] Registering TS with master...
I20260926 11:44:12.242736 23721 heartbeater.cc:511] Master 127.6.96.62:40223 requested a full tablet report, sending...
I20260926 11:44:12.244223 23530 ts_manager.cc:194] Registered new tserver with Master: 49ba1d8b81c94e6d91175975d65db5ce (127.6.96.1:45135)
WARNING: no policy specified for kudu/127.6.96.2@KRBTEST.COM; defaulting to no policy
I20260926 11:44:12.245337 23530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.1@KRBTEST.COM'} at 127.6.96.1:56611
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
I20260926 11:44:12.262180 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:40223
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:42645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:12.372910 23726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:12.373183 23726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:12.373224 23726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:12.377377 23726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:12.377446 23726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:12.377512 23726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:44:12.382862 23726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:42645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:40223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:12.383919 23726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:12.384955 23726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:12.394536 23732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:12.394536 23731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:12.394577 23726 server_base.cc:1034] running on GCE node
W20260926 11:44:12.394536 23734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:12.395246 23726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:12.395877 23726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:12.397040 23726 hybrid_clock.cc:648] HybridClock initialized: now 1790423052397024 us; error 32 us; skew 500 ppm
Sep 26 11:44:12 dist-test-slave-xxxx krb5kdc[23485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423052, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:12.400064 23726 init.cc:377] Logged in from keytab as kudu/127.6.96.2@KRBTEST.COM (short username kudu)
I20260926 11:44:12.401055 23726 webserver.cc:460] Webserver started at http://127.6.96.2:35311/ using document root <none> and password file <none>
I20260926 11:44:12.401643 23726 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:12.401706 23726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:12.401932 23726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:12.404030 23726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "55d718825d484069a32d9c57a02a828d"
format_stamp: "Formatted at 2026-09-26 11:44:12 on dist-test-slave-xxxx"
server_key: "f535c2b969f1028b62b69b65ff0279e1"
server_key_iv: "ce1ff5fd3c05ccbe00523577f5b3cb23"
server_key_version: "encryptionkey@0"
I20260926 11:44:12.404561 23726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "55d718825d484069a32d9c57a02a828d"
format_stamp: "Formatted at 2026-09-26 11:44:12 on dist-test-slave-xxxx"
server_key: "f535c2b969f1028b62b69b65ff0279e1"
server_key_iv: "ce1ff5fd3c05ccbe00523577f5b3cb23"
server_key_version: "encryptionkey@0"
I20260926 11:44:12.408337 23726 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 11:44:12.410760 23741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:12.412034 23726 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:44:12.412179 23726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "55d718825d484069a32d9c57a02a828d"
format_stamp: "Formatted at 2026-09-26 11:44:12 on dist-test-slave-xxxx"
server_key: "f535c2b969f1028b62b69b65ff0279e1"
server_key_iv: "ce1ff5fd3c05ccbe00523577f5b3cb23"
server_key_version: "encryptionkey@0"
I20260926 11:44:12.412284 23726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:12.424274 23726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:12.427450 23726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:12.427631 23726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:12.428263 23726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:44:12.429104 23726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:44:12.429152 23726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:12.429214 23726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:44:12.429242 23726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:12.437877 23726 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:40863
I20260926 11:44:12.437915 23854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:40863 every 8 connection(s)
I20260926 11:44:12.438854 23726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:44:12.448289 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 23726
I20260926 11:44:12.448411 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
Sep 26 11:44:12 dist-test-slave-xxxx krb5kdc[23485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423052, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:44:12.448700 6528 external_mini_cluster.cc:1428] Setting key df1fe89343db28a1489cb14fd52853cb
I20260926 11:44:12.452935 23569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:12.440676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:34357 (local address 127.6.96.62:40223)
0926 11:44:12.440798 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:12.440802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:12.441667 (+ 865us) server_negotiation.cc:407] Connection header received
0926 11:44:12.443144 (+ 1477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:12.443148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:12.443245 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:12.443315 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:12.444641 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.445163 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.446327 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.446546 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.446632 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:12.449292 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:12.449296 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:12.449299 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:12.449328 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:12.450932 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:12.451438 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:12.451441 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:12.451443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:12.451529 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:12.451859 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:12.451863 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:12.451865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:12.452056 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:12.452145 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:12.452653 (+ 508us) server_negotiation.cc:299] Negotiation successful
0926 11:44:12.452778 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 11:44:12.453763 23857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:12.441004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:40223 (local address 127.6.96.2:34357)
0926 11:44:12.441498 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:12.441542 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:12.442770 (+ 1228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:12.443531 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:12.443538 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:12.443972 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:12.444485 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:12.444499 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.445293 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.445296 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:12.446202 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:12.446211 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.446699 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.446704 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:12.446910 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:12.447484 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:12.447499 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:12.449063 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:12.451044 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:12.451049 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:12.451063 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:12.451342 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:12.451608 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:12.451610 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:12.451611 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:12.451767 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:12.452158 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:12.452165 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:12.452424 (+ 259us) client_negotiation.cc:769] Sending connection context
0926 11:44:12.452596 (+ 172us) client_negotiation.cc:240] Negotiation successful
0926 11:44:12.452943 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":111,"threads_started":1}
I20260926 11:44:12.454877 23855 heartbeater.cc:347] Connected to a master server at 127.6.96.62:40223
I20260926 11:44:12.455147 23855 heartbeater.cc:464] Registering TS with master...
I20260926 11:44:12.455750 23855 heartbeater.cc:511] Master 127.6.96.62:40223 requested a full tablet report, sending...
I20260926 11:44:12.456876 23530 ts_manager.cc:194] Registered new tserver with Master: 55d718825d484069a32d9c57a02a828d (127.6.96.2:40863)
I20260926 11:44:12.457476 23530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.2@KRBTEST.COM'} at 127.6.96.2:34357
WARNING: no policy specified for kudu/127.6.96.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
I20260926 11:44:12.477563 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:40223
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:42645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:12.588083 23860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:12.588357 23860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:12.588407 23860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:12.592612 23860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:12.592690 23860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:12.592775 23860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:44:12.598300 23860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:42645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:40223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:12.599342 23860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:12.600288 23860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:12.609860 23866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:12.609875 23865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:12.609860 23868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:12.610080 23860 server_base.cc:1034] running on GCE node
I20260926 11:44:12.610605 23860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:12.611292 23860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:12.612455 23860 hybrid_clock.cc:648] HybridClock initialized: now 1790423052612436 us; error 33 us; skew 500 ppm
Sep 26 11:44:12 dist-test-slave-xxxx krb5kdc[23485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423052, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:12.619508 23860 init.cc:377] Logged in from keytab as kudu/127.6.96.3@KRBTEST.COM (short username kudu)
I20260926 11:44:12.620501 23860 webserver.cc:460] Webserver started at http://127.6.96.3:36879/ using document root <none> and password file <none>
I20260926 11:44:12.621065 23860 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:12.621133 23860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:12.621331 23860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:12.623514 23860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "2b8ce8ed96b74449b05f70ec7e7f7dcf"
format_stamp: "Formatted at 2026-09-26 11:44:12 on dist-test-slave-xxxx"
server_key: "822a141c443f99b48d89dd43f33957f1"
server_key_iv: "6ed7707d85fabc67ea6574221d1542da"
server_key_version: "encryptionkey@0"
I20260926 11:44:12.624089 23860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "2b8ce8ed96b74449b05f70ec7e7f7dcf"
format_stamp: "Formatted at 2026-09-26 11:44:12 on dist-test-slave-xxxx"
server_key: "822a141c443f99b48d89dd43f33957f1"
server_key_iv: "6ed7707d85fabc67ea6574221d1542da"
server_key_version: "encryptionkey@0"
I20260926 11:44:12.628028 23860 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260926 11:44:12.630483 23875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:12.631825 23860 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:44:12.631955 23860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "2b8ce8ed96b74449b05f70ec7e7f7dcf"
format_stamp: "Formatted at 2026-09-26 11:44:12 on dist-test-slave-xxxx"
server_key: "822a141c443f99b48d89dd43f33957f1"
server_key_iv: "6ed7707d85fabc67ea6574221d1542da"
server_key_version: "encryptionkey@0"
I20260926 11:44:12.632064 23860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:12.647181 23860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:12.649883 23860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:12.650063 23860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:12.650620 23860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:44:12.651454 23860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:44:12.651500 23860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:12.651562 23860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:44:12.651587 23860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:12.660197 23860 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:35187
I20260926 11:44:12.660223 23988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:35187 every 8 connection(s)
I20260926 11:44:12.661090 23860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:44:12.663947 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 23860
I20260926 11:44:12.664126 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:44:12.664467 6528 external_mini_cluster.cc:1428] Setting key a8003e366e15b39ea7a3f769d9137ddb
Sep 26 11:44:12 dist-test-slave-xxxx krb5kdc[23485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423052, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:44:12.674417 23569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:12.662862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:52847 (local address 127.6.96.62:40223)
0926 11:44:12.663020 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:12.663024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:12.663924 (+ 900us) server_negotiation.cc:407] Connection header received
0926 11:44:12.665368 (+ 1444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:12.665372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:12.665427 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:12.665499 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:12.666863 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.667402 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.668502 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.668717 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.668801 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:12.671145 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:12.671148 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:12.671150 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:12.671174 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:12.672410 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:12.672923 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:12.672926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:12.672927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:12.673002 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:12.673316 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:12.673319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:12.673321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:12.673499 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:12.673585 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:12.674137 (+ 552us) server_negotiation.cc:299] Negotiation successful
0926 11:44:12.674255 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 11:44:12.675372 23991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:12.663270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:40223 (local address 127.6.96.3:52847)
0926 11:44:12.663744 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:12.663803 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:12.665154 (+ 1351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:12.665692 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:12.665698 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:12.666162 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:12.666714 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:12.666728 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.667523 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.667526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:12.668392 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:12.668401 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.668847 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.668850 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:12.669012 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:12.669516 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:12.669555 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:12.670984 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:12.672545 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:12.672551 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:12.672565 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:12.672825 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:12.673092 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:12.673096 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:12.673098 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:12.673231 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:12.673598 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:12.673605 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:12.673912 (+ 307us) client_negotiation.cc:769] Sending connection context
0926 11:44:12.674083 (+ 171us) client_negotiation.cc:240] Negotiation successful
0926 11:44:12.674423 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":100,"threads_started":1}
I20260926 11:44:12.676509 23989 heartbeater.cc:347] Connected to a master server at 127.6.96.62:40223
I20260926 11:44:12.676769 23989 heartbeater.cc:464] Registering TS with master...
I20260926 11:44:12.677330 23989 heartbeater.cc:511] Master 127.6.96.62:40223 requested a full tablet report, sending...
I20260926 11:44:12.678357 23530 ts_manager.cc:194] Registered new tserver with Master: 2b8ce8ed96b74449b05f70ec7e7f7dcf (127.6.96.3:35187)
I20260926 11:44:12.678988 23530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.3@KRBTEST.COM'} at 127.6.96.3:52847
I20260926 11:44:12.690478 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 11:44:12.699332 23569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:12.692489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49128 (local address 127.6.96.62:40223)
0926 11:44:12.692639 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:12.692644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:12.692773 (+ 129us) server_negotiation.cc:407] Connection header received
0926 11:44:12.692897 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:12.692900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:12.692945 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:12.693008 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:12.693686 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.694206 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.695030 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.695254 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.695339 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:12.696576 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:12.696580 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:12.696583 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:12.696608 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:12.697798 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:12.698204 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:12.698208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:12.698210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:12.698290 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:12.698565 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:12.698567 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:12.698568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:12.698771 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:12.698907 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:12.699083 (+ 176us) server_negotiation.cc:299] Negotiation successful
0926 11:44:12.699179 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 11:44:12.702098 23530 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49128:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 11:44:12.705188 23530 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:44:12.719167 24000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:12.713267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:45135 (local address 127.0.0.1:41132)
0926 11:44:12.714029 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:12.714043 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:12.714112 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:12.715164 (+ 1052us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:12.715167 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:12.715182 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:12.715283 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:12.715289 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.716359 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.716362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:12.717544 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:12.717553 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.718640 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.718644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:12.718751 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:12.718794 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 11:44:12.718897 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 11:44:12.719013 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":80,"threads_started":1}
I20260926 11:44:12.719568 24001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:12.713226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:35187 (local address 127.0.0.1:49406)
0926 11:44:12.713618 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:12.713668 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:12.713808 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:12.714955 (+ 1147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:12.714961 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:12.714984 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:12.715132 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:12.715139 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.716765 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.716768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:12.717983 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:12.717995 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.719163 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.719167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:12.719287 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:12.719313 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 11:44:12.719363 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 11:44:12.719431 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":28,"thread_start_us":98,"threads_started":1}
I20260926 11:44:12.720305 24002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:12.713342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49406 (local address 127.6.96.3:35187)
0926 11:44:12.714465 (+ 1123us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:12.714477 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:12.714498 (+ 21us) server_negotiation.cc:407] Connection header received
0926 11:44:12.714577 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:12.714582 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:12.714791 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:12.714971 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:12.715325 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.716617 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.718128 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.719018 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.719831 (+ 813us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:12.719986 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:12.720038 (+ 52us) server_negotiation.cc:299] Negotiation successful
0926 11:44:12.720131 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1030,"thread_start_us":152,"threads_started":1}
I20260926 11:44:12.720556 24004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:12.714046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41132 (local address 127.6.96.1:45135)
0926 11:44:12.714313 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:12.714328 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:12.714351 (+ 23us) server_negotiation.cc:407] Connection header received
0926 11:44:12.714601 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:12.714606 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:12.714791 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:12.714971 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:12.715406 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.716205 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.717663 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.718495 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.719362 (+ 867us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:12.719519 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:12.719583 (+ 64us) server_negotiation.cc:299] Negotiation successful
0926 11:44:12.719685 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":94,"threads_started":1}
I20260926 11:44:12.721958 24003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:12.713942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:40863 (local address 127.0.0.1:36036)
0926 11:44:12.714541 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:12.714552 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:12.714624 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:12.715298 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:12.715301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:12.715315 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:12.715390 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:12.715396 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.716734 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.716737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:12.719002 (+ 2265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:12.719033 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.721586 (+ 2553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.721589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:12.721676 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:12.721700 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 11:44:12.721766 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 11:44:12.721835 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":71,"thread_start_us":83,"threads_started":1}
I20260926 11:44:12.722409 24005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:12.714377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36036 (local address 127.6.96.2:40863)
0926 11:44:12.714851 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:12.714867 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:12.714887 (+ 20us) server_negotiation.cc:407] Connection header received
0926 11:44:12.714957 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:12.714961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:12.715144 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:12.715312 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:12.715459 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.716585 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.719222 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:12.720614 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:12.721388 (+ 774us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:12.721523 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:12.722171 (+ 648us) server_negotiation.cc:299] Negotiation successful
0926 11:44:12.722263 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":106,"threads_started":1}
I20260926 11:44:12.723613 23655 tablet_service.cc:1467] Processing CreateTablet for tablet f8109349a29b487685a496da48fefa35 (DEFAULT_TABLE table=test-table [id=49ec811f30f24e55b8c0c29171e1a06e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:44:12.724737 23655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8109349a29b487685a496da48fefa35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:12.724974 23789 tablet_service.cc:1467] Processing CreateTablet for tablet f8109349a29b487685a496da48fefa35 (DEFAULT_TABLE table=test-table [id=49ec811f30f24e55b8c0c29171e1a06e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:44:12.725011 23923 tablet_service.cc:1467] Processing CreateTablet for tablet f8109349a29b487685a496da48fefa35 (DEFAULT_TABLE table=test-table [id=49ec811f30f24e55b8c0c29171e1a06e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:44:12.726094 23789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8109349a29b487685a496da48fefa35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:12.726291 23923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8109349a29b487685a496da48fefa35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:12.731870 24006 tablet_bootstrap.cc:492] T f8109349a29b487685a496da48fefa35 P 49ba1d8b81c94e6d91175975d65db5ce: Bootstrap starting.
I20260926 11:44:12.733670 24008 tablet_bootstrap.cc:492] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf: Bootstrap starting.
I20260926 11:44:12.734083 24007 tablet_bootstrap.cc:492] T f8109349a29b487685a496da48fefa35 P 55d718825d484069a32d9c57a02a828d: Bootstrap starting.
I20260926 11:44:12.734138 24006 tablet_bootstrap.cc:654] T f8109349a29b487685a496da48fefa35 P 49ba1d8b81c94e6d91175975d65db5ce: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:12.735654 24006 log.cc:824] T f8109349a29b487685a496da48fefa35 P 49ba1d8b81c94e6d91175975d65db5ce: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:12.736100 24007 tablet_bootstrap.cc:654] T f8109349a29b487685a496da48fefa35 P 55d718825d484069a32d9c57a02a828d: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:12.736142 24008 tablet_bootstrap.cc:654] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:12.737138 24007 log.cc:824] T f8109349a29b487685a496da48fefa35 P 55d718825d484069a32d9c57a02a828d: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:12.737236 24008 log.cc:824] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:12.737881 24006 tablet_bootstrap.cc:492] T f8109349a29b487685a496da48fefa35 P 49ba1d8b81c94e6d91175975d65db5ce: No bootstrap required, opened a new log
I20260926 11:44:12.738076 24006 ts_tablet_manager.cc:1397] T f8109349a29b487685a496da48fefa35 P 49ba1d8b81c94e6d91175975d65db5ce: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 11:44:12.738816 24008 tablet_bootstrap.cc:492] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf: No bootstrap required, opened a new log
I20260926 11:44:12.739029 24007 tablet_bootstrap.cc:492] T f8109349a29b487685a496da48fefa35 P 55d718825d484069a32d9c57a02a828d: No bootstrap required, opened a new log
I20260926 11:44:12.739023 24008 ts_tablet_manager.cc:1397] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260926 11:44:12.739221 24007 ts_tablet_manager.cc:1397] T f8109349a29b487685a496da48fefa35 P 55d718825d484069a32d9c57a02a828d: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.004s
I20260926 11:44:12.742015 24008 raft_consensus.cc:348] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8ce8ed96b74449b05f70ec7e7f7dcf" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35187 } } peers { permanent_uuid: "49ba1d8b81c94e6d91175975d65db5ce" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 45135 } } peers { permanent_uuid: "55d718825d484069a32d9c57a02a828d" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 40863 } }
I20260926 11:44:12.742259 24008 raft_consensus.cc:374] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:12.742120 24006 raft_consensus.cc:348] T f8109349a29b487685a496da48fefa35 P 49ba1d8b81c94e6d91175975d65db5ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8ce8ed96b74449b05f70ec7e7f7dcf" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35187 } } peers { permanent_uuid: "49ba1d8b81c94e6d91175975d65db5ce" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 45135 } } peers { permanent_uuid: "55d718825d484069a32d9c57a02a828d" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 40863 } }
I20260926 11:44:12.742328 24008 raft_consensus.cc:729] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b8ce8ed96b74449b05f70ec7e7f7dcf, State: Initialized, Role: FOLLOWER
I20260926 11:44:12.742398 24006 raft_consensus.cc:374] T f8109349a29b487685a496da48fefa35 P 49ba1d8b81c94e6d91175975d65db5ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:12.742458 24006 raft_consensus.cc:729] T f8109349a29b487685a496da48fefa35 P 49ba1d8b81c94e6d91175975d65db5ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49ba1d8b81c94e6d91175975d65db5ce, State: Initialized, Role: FOLLOWER
I20260926 11:44:12.742744 24008 consensus_queue.cc:260] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8ce8ed96b74449b05f70ec7e7f7dcf" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35187 } } peers { permanent_uuid: "49ba1d8b81c94e6d91175975d65db5ce" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 45135 } } peers { permanent_uuid: "55d718825d484069a32d9c57a02a828d" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 40863 } }
I20260926 11:44:12.742786 24007 raft_consensus.cc:348] T f8109349a29b487685a496da48fefa35 P 55d718825d484069a32d9c57a02a828d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8ce8ed96b74449b05f70ec7e7f7dcf" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35187 } } peers { permanent_uuid: "49ba1d8b81c94e6d91175975d65db5ce" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 45135 } } peers { permanent_uuid: "55d718825d484069a32d9c57a02a828d" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 40863 } }
I20260926 11:44:12.742957 24006 consensus_queue.cc:260] T f8109349a29b487685a496da48fefa35 P 49ba1d8b81c94e6d91175975d65db5ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8ce8ed96b74449b05f70ec7e7f7dcf" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35187 } } peers { permanent_uuid: "49ba1d8b81c94e6d91175975d65db5ce" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 45135 } } peers { permanent_uuid: "55d718825d484069a32d9c57a02a828d" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 40863 } }
I20260926 11:44:12.743065 24007 raft_consensus.cc:374] T f8109349a29b487685a496da48fefa35 P 55d718825d484069a32d9c57a02a828d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:12.743160 24007 raft_consensus.cc:729] T f8109349a29b487685a496da48fefa35 P 55d718825d484069a32d9c57a02a828d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55d718825d484069a32d9c57a02a828d, State: Initialized, Role: FOLLOWER
I20260926 11:44:12.743465 23989 heartbeater.cc:503] Master 127.6.96.62:40223 was elected leader, sending a full tablet report...
I20260926 11:44:12.743712 24008 ts_tablet_manager.cc:1428] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260926 11:44:12.743826 24006 ts_tablet_manager.cc:1428] T f8109349a29b487685a496da48fefa35 P 49ba1d8b81c94e6d91175975d65db5ce: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 11:44:12.744124 23721 heartbeater.cc:503] Master 127.6.96.62:40223 was elected leader, sending a full tablet report...
I20260926 11:44:12.744640 24007 consensus_queue.cc:260] T f8109349a29b487685a496da48fefa35 P 55d718825d484069a32d9c57a02a828d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8ce8ed96b74449b05f70ec7e7f7dcf" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35187 } } peers { permanent_uuid: "49ba1d8b81c94e6d91175975d65db5ce" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 45135 } } peers { permanent_uuid: "55d718825d484069a32d9c57a02a828d" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 40863 } }
I20260926 11:44:12.745412 24007 ts_tablet_manager.cc:1428] T f8109349a29b487685a496da48fefa35 P 55d718825d484069a32d9c57a02a828d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 11:44:12.746743 23855 heartbeater.cc:503] Master 127.6.96.62:40223 was elected leader, sending a full tablet report...
W20260926 11:44:12.912019 23990 tablet.cc:2369] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 11:44:12.939994 23856 tablet.cc:2369] T f8109349a29b487685a496da48fefa35 P 55d718825d484069a32d9c57a02a828d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 11:44:12.977459 23722 tablet.cc:2369] T f8109349a29b487685a496da48fefa35 P 49ba1d8b81c94e6d91175975d65db5ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:44:12.997555 24012 raft_consensus.cc:482] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:44:12.997758 24012 raft_consensus.cc:504] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8ce8ed96b74449b05f70ec7e7f7dcf" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35187 } } peers { permanent_uuid: "49ba1d8b81c94e6d91175975d65db5ce" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 45135 } } peers { permanent_uuid: "55d718825d484069a32d9c57a02a828d" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 40863 } }
I20260926 11:44:12.998912 24012 leader_election.cc:288] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49ba1d8b81c94e6d91175975d65db5ce (127.6.96.1:45135), 55d718825d484069a32d9c57a02a828d (127.6.96.2:40863)
I20260926 11:44:13.003618 24005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:12.999362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:44803 (local address 127.6.96.2:40863)
0926 11:44:12.999468 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:12.999473 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:12.999669 (+ 196us) server_negotiation.cc:407] Connection header received
0926 11:44:12.999744 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:12.999747 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:12.999789 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:12.999858 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:13.000108 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.000824 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.002614 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.003089 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.003172 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:13.003234 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:13.003402 (+ 168us) server_negotiation.cc:299] Negotiation successful
0926 11:44:13.003497 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 11:44:13.003600 24015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:12.999180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:40863 (local address 127.6.96.3:44803)
0926 11:44:12.999592 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:12.999604 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:12.999677 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:12.999872 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:12.999875 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:12.999888 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:13.000010 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:13.000016 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.000948 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.000952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:13.002501 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:13.002507 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.003195 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.003199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:13.003295 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:13.003318 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 11:44:13.003376 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 11:44:13.003437 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":227,"thread_start_us":110,"threads_started":1}
I20260926 11:44:13.003666 24004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:12.999276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:41471 (local address 127.6.96.1:45135)
0926 11:44:12.999428 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:12.999433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:12.999451 (+ 18us) server_negotiation.cc:407] Connection header received
0926 11:44:12.999603 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:12.999605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:12.999650 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:12.999749 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:13.000024 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.000814 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.002652 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.003101 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.003197 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:13.003257 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:13.003480 (+ 223us) server_negotiation.cc:299] Negotiation successful
0926 11:44:13.003564 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 11:44:13.003623 23991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:12.999180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:45135 (local address 127.6.96.3:41471)
0926 11:44:12.999380 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:12.999396 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:12.999493 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:12.999767 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:12.999769 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:12.999782 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:12.999914 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:12.999921 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.000948 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.000952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:13.002492 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:13.002504 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.003231 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.003235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:13.003354 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:13.003376 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 11:44:13.003448 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 11:44:13.003518 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":116}
I20260926 11:44:13.004268 23809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8109349a29b487685a496da48fefa35" candidate_uuid: "2b8ce8ed96b74449b05f70ec7e7f7dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55d718825d484069a32d9c57a02a828d" is_pre_election: true
I20260926 11:44:13.004266 23675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8109349a29b487685a496da48fefa35" candidate_uuid: "2b8ce8ed96b74449b05f70ec7e7f7dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49ba1d8b81c94e6d91175975d65db5ce" is_pre_election: true
I20260926 11:44:13.004575 23675 raft_consensus.cc:2454] T f8109349a29b487685a496da48fefa35 P 49ba1d8b81c94e6d91175975d65db5ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b8ce8ed96b74449b05f70ec7e7f7dcf in term 0.
I20260926 11:44:13.004575 23809 raft_consensus.cc:2454] T f8109349a29b487685a496da48fefa35 P 55d718825d484069a32d9c57a02a828d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b8ce8ed96b74449b05f70ec7e7f7dcf in term 0.
I20260926 11:44:13.005119 23879 leader_election.cc:302] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b8ce8ed96b74449b05f70ec7e7f7dcf, 49ba1d8b81c94e6d91175975d65db5ce; no voters:
I20260926 11:44:13.005393 24012 raft_consensus.cc:2781] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:44:13.005475 24012 raft_consensus.cc:482] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:44:13.005543 24012 raft_consensus.cc:3037] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:13.006640 24012 raft_consensus.cc:504] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8ce8ed96b74449b05f70ec7e7f7dcf" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35187 } } peers { permanent_uuid: "49ba1d8b81c94e6d91175975d65db5ce" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 45135 } } peers { permanent_uuid: "55d718825d484069a32d9c57a02a828d" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 40863 } }
I20260926 11:44:13.007017 24012 leader_election.cc:288] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf [CANDIDATE]: Term 1 election: Requested vote from peers 49ba1d8b81c94e6d91175975d65db5ce (127.6.96.1:45135), 55d718825d484069a32d9c57a02a828d (127.6.96.2:40863)
I20260926 11:44:13.007400 23675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8109349a29b487685a496da48fefa35" candidate_uuid: "2b8ce8ed96b74449b05f70ec7e7f7dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49ba1d8b81c94e6d91175975d65db5ce"
I20260926 11:44:13.007400 23809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8109349a29b487685a496da48fefa35" candidate_uuid: "2b8ce8ed96b74449b05f70ec7e7f7dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55d718825d484069a32d9c57a02a828d"
I20260926 11:44:13.007529 23809 raft_consensus.cc:3037] T f8109349a29b487685a496da48fefa35 P 55d718825d484069a32d9c57a02a828d [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:13.007529 23675 raft_consensus.cc:3037] T f8109349a29b487685a496da48fefa35 P 49ba1d8b81c94e6d91175975d65db5ce [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:13.008756 23809 raft_consensus.cc:2454] T f8109349a29b487685a496da48fefa35 P 55d718825d484069a32d9c57a02a828d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b8ce8ed96b74449b05f70ec7e7f7dcf in term 1.
I20260926 11:44:13.008756 23675 raft_consensus.cc:2454] T f8109349a29b487685a496da48fefa35 P 49ba1d8b81c94e6d91175975d65db5ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b8ce8ed96b74449b05f70ec7e7f7dcf in term 1.
I20260926 11:44:13.009107 23879 leader_election.cc:302] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2b8ce8ed96b74449b05f70ec7e7f7dcf, 49ba1d8b81c94e6d91175975d65db5ce, 55d718825d484069a32d9c57a02a828d; no voters:
I20260926 11:44:13.009303 24012 raft_consensus.cc:2781] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:13.009536 24012 raft_consensus.cc:686] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf [term 1 LEADER]: Becoming Leader. State: Replica: 2b8ce8ed96b74449b05f70ec7e7f7dcf, State: Running, Role: LEADER
I20260926 11:44:13.009876 24012 consensus_queue.cc:237] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8ce8ed96b74449b05f70ec7e7f7dcf" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35187 } } peers { permanent_uuid: "49ba1d8b81c94e6d91175975d65db5ce" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 45135 } } peers { permanent_uuid: "55d718825d484069a32d9c57a02a828d" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 40863 } }
I20260926 11:44:13.012940 23528 catalog_manager.cc:5495] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b8ce8ed96b74449b05f70ec7e7f7dcf (127.6.96.3). New cstate: current_term: 1 leader_uuid: "2b8ce8ed96b74449b05f70ec7e7f7dcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8ce8ed96b74449b05f70ec7e7f7dcf" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 35187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49ba1d8b81c94e6d91175975d65db5ce" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55d718825d484069a32d9c57a02a828d" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 40863 } health_report { overall_health: UNKNOWN } } }
I20260926 11:44:13.081537 24002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:13.078050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49408 (local address 127.6.96.3:35187)
0926 11:44:13.078181 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:13.078185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:13.078200 (+ 15us) server_negotiation.cc:407] Connection header received
0926 11:44:13.078352 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:13.078354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:13.078408 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:13.078492 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:44:13.078729 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.079303 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.080153 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.080370 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.080454 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:13.080735 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:44:13.081150 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:44:13.081254 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:13.081318 (+ 64us) server_negotiation.cc:299] Negotiation successful
0926 11:44:13.081375 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 11:44:13.087497 23675 raft_consensus.cc:1261] T f8109349a29b487685a496da48fefa35 P 49ba1d8b81c94e6d91175975d65db5ce [term 1 FOLLOWER]: Refusing update from remote peer 2b8ce8ed96b74449b05f70ec7e7f7dcf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:44:13.087497 23809 raft_consensus.cc:1261] T f8109349a29b487685a496da48fefa35 P 55d718825d484069a32d9c57a02a828d [term 1 FOLLOWER]: Refusing update from remote peer 2b8ce8ed96b74449b05f70ec7e7f7dcf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:44:13.088277 24016 consensus_queue.cc:1035] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "55d718825d484069a32d9c57a02a828d" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 40863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:44:13.088455 24012 consensus_queue.cc:1035] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "49ba1d8b81c94e6d91175975d65db5ce" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:44:13.095781 24019 mvcc.cc:205] Tried to move back new op lower bound from 7333572825438343168 to 7333572825131167744. Current Snapshot: MvccSnapshot[applied={T|T < 7333572825438343168}]
I20260926 11:44:13.096444 24021 mvcc.cc:205] Tried to move back new op lower bound from 7333572825438343168 to 7333572825131167744. Current Snapshot: MvccSnapshot[applied={T|T < 7333572825438343168}]
I20260926 11:44:13.106901 23528 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:44:13.107086 23528 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:44:13.109350 23528 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b20543e9ea64587984bdb98f72e9f6f: Sending DeleteTablet for 3 replicas of tablet f8109349a29b487685a496da48fefa35
I20260926 11:44:13.110134 23923 tablet_service.cc:1514] Processing DeleteTablet for tablet f8109349a29b487685a496da48fefa35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:44:13 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:49406
I20260926 11:44:13.110164 23655 tablet_service.cc:1514] Processing DeleteTablet for tablet f8109349a29b487685a496da48fefa35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:44:13 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:41132
I20260926 11:44:13.110350 23789 tablet_service.cc:1514] Processing DeleteTablet for tablet f8109349a29b487685a496da48fefa35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:44:13 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:36036
I20260926 11:44:13.110673 24029 tablet_replica.cc:330] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf: stopping tablet replica
I20260926 11:44:13.111012 24029 raft_consensus.cc:2229] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf [term 1 LEADER]: Raft consensus shutting down.
I20260926 11:44:13.111116 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 23592
I20260926 11:44:13.112740 24029 raft_consensus.cc:2258] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:44:13.114490 24029 ts_tablet_manager.cc:1905] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:44:13.115062 24031 tablet_replica.cc:330] T f8109349a29b487685a496da48fefa35 P 55d718825d484069a32d9c57a02a828d: stopping tablet replica
I20260926 11:44:13.115525 24031 raft_consensus.cc:2229] T f8109349a29b487685a496da48fefa35 P 55d718825d484069a32d9c57a02a828d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:44:13.115959 24031 raft_consensus.cc:2258] T f8109349a29b487685a496da48fefa35 P 55d718825d484069a32d9c57a02a828d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:44:13.117453 24029 ts_tablet_manager.cc:1918] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:44:13.117558 24029 log.cc:1196] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/f8109349a29b487685a496da48fefa35
W20260926 11:44:13.117784 23517 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.96.1:45135 (error 108)
I20260926 11:44:13.117872 24029 ts_tablet_manager.cc:1939] T f8109349a29b487685a496da48fefa35 P 2b8ce8ed96b74449b05f70ec7e7f7dcf: Deleting consensus metadata
I20260926 11:44:13.117836 24031 ts_tablet_manager.cc:1905] T f8109349a29b487685a496da48fefa35 P 55d718825d484069a32d9c57a02a828d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:44:13.119279 24003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:13.118935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:45135 (local address 127.0.0.1:41136)
0926 11:44:13.119107 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:13.119174 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.1:45135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260926 11:44:13.119557 23516 catalog_manager.cc:4841] TS 2b8ce8ed96b74449b05f70ec7e7f7dcf (127.6.96.3:35187): tablet f8109349a29b487685a496da48fefa35 (table test-table [id=49ec811f30f24e55b8c0c29171e1a06e]) successfully deleted
W20260926 11:44:13.119589 23517 catalog_manager.cc:4584] TS 49ba1d8b81c94e6d91175975d65db5ce (127.6.96.1:45135): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f8109349a29b487685a496da48fefa35: Network error: Client connection negotiation failed: client connection to 127.6.96.1:45135: connect: Connection refused (error 111)
I20260926 11:44:13.119910 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 23726
I20260926 11:44:13.125398 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 23860
I20260926 11:44:13.126415 24003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:13.125639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:40863 (local address 127.0.0.1:36038)
0926 11:44:13.126237 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:13.126289 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.2:40863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519}
I20260926 11:44:13.130306 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 23498
2026-09-26T11:44:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2425 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-26T11:44:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:13Z Disabled control of system clock
I20260926 11:44:13.157848 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:38225
--webserver_interface=127.6.96.62
--webserver_port=0
--builtin_ntp_servers=127.6.96.20:37051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:38225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 11:44:13.266278 24036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:13.266541 24036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:13.266575 24036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:13.270538 24036 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:13.270602 24036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:13.270617 24036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:13.270632 24036 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:13.270645 24036 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:13.275483 24036 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:37051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:38225
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:38225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:13.276518 24036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:13.277812 24036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:13.285817 24044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:13.285831 24041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:13.285840 24042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:13.286058 24036 server_base.cc:1034] running on GCE node
I20260926 11:44:13.286662 24036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:13.287794 24036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:13.288955 24036 hybrid_clock.cc:648] HybridClock initialized: now 1790423053288943 us; error 60 us; skew 500 ppm
I20260926 11:44:13.290762 24036 webserver.cc:460] Webserver started at http://127.6.96.62:34205/ using document root <none> and password file <none>
I20260926 11:44:13.291349 24036 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:13.291417 24036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:13.291616 24036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:13.293684 24036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "1aad3023ce0c41e982be53c5b7ad1ba1"
format_stamp: "Formatted at 2026-09-26 11:44:13 on dist-test-slave-xxxx"
server_key: "cb67f728477ab43d4f70bc3e574835e4"
server_key_iv: "805e2b7a1b9c9917fb76dc2101c3bdd1"
server_key_version: "encryptionkey@0"
I20260926 11:44:13.294229 24036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "1aad3023ce0c41e982be53c5b7ad1ba1"
format_stamp: "Formatted at 2026-09-26 11:44:13 on dist-test-slave-xxxx"
server_key: "cb67f728477ab43d4f70bc3e574835e4"
server_key_iv: "805e2b7a1b9c9917fb76dc2101c3bdd1"
server_key_version: "encryptionkey@0"
I20260926 11:44:13.297806 24036 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 11:44:13.300387 24050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:13.301635 24036 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 11:44:13.301761 24036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "1aad3023ce0c41e982be53c5b7ad1ba1"
format_stamp: "Formatted at 2026-09-26 11:44:13 on dist-test-slave-xxxx"
server_key: "cb67f728477ab43d4f70bc3e574835e4"
server_key_iv: "805e2b7a1b9c9917fb76dc2101c3bdd1"
server_key_version: "encryptionkey@0"
I20260926 11:44:13.301862 24036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:13.319860 24036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:13.320596 24036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:13.320773 24036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:13.327550 24036 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:38225
I20260926 11:44:13.327572 24102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:38225 every 8 connection(s)
I20260926 11:44:13.328538 24036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:13.331772 24103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:13.333463 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 24036
I20260926 11:44:13.333576 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:13.333861 6528 external_mini_cluster.cc:1428] Setting key e14ddd026d509e17655a96147d621fce
I20260926 11:44:13.337805 24103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aad3023ce0c41e982be53c5b7ad1ba1: Bootstrap starting.
I20260926 11:44:13.340349 24103 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1aad3023ce0c41e982be53c5b7ad1ba1: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:13.341173 24103 log.cc:824] T 00000000000000000000000000000000 P 1aad3023ce0c41e982be53c5b7ad1ba1: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:13.343448 24103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aad3023ce0c41e982be53c5b7ad1ba1: No bootstrap required, opened a new log
I20260926 11:44:13.343601 24106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:13.335274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54352 (local address 127.6.96.62:38225)
0926 11:44:13.336042 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:13.336062 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:13.336100 (+ 38us) server_negotiation.cc:407] Connection header received
0926 11:44:13.336881 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:13.336902 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:13.337335 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:13.337626 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:13.338004 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.339038 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.340219 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.340731 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.340850 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:13.341699 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:13.341706 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:13.341723 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:13.342183 (+ 460us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:13.342254 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:13.342275 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:13.342350 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:13.342522 (+ 172us) server_negotiation.cc:299] Negotiation successful
0926 11:44:13.342786 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":110,"threads_started":1}
I20260926 11:44:13.347334 24103 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1aad3023ce0c41e982be53c5b7ad1ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aad3023ce0c41e982be53c5b7ad1ba1" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 38225 } }
I20260926 11:44:13.347570 24103 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1aad3023ce0c41e982be53c5b7ad1ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:13.347625 24103 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1aad3023ce0c41e982be53c5b7ad1ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aad3023ce0c41e982be53c5b7ad1ba1, State: Initialized, Role: FOLLOWER
I20260926 11:44:13.348017 24103 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1aad3023ce0c41e982be53c5b7ad1ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aad3023ce0c41e982be53c5b7ad1ba1" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 38225 } }
I20260926 11:44:13.348150 24103 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1aad3023ce0c41e982be53c5b7ad1ba1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:44:13.348212 24103 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1aad3023ce0c41e982be53c5b7ad1ba1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:44:13.348308 24103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1aad3023ce0c41e982be53c5b7ad1ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:13.349220 24103 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1aad3023ce0c41e982be53c5b7ad1ba1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aad3023ce0c41e982be53c5b7ad1ba1" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 38225 } }
I20260926 11:44:13.349539 24103 leader_election.cc:302] T 00000000000000000000000000000000 P 1aad3023ce0c41e982be53c5b7ad1ba1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1aad3023ce0c41e982be53c5b7ad1ba1; no voters:
I20260926 11:44:13.349826 24103 leader_election.cc:288] T 00000000000000000000000000000000 P 1aad3023ce0c41e982be53c5b7ad1ba1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:44:13.350005 24108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1aad3023ce0c41e982be53c5b7ad1ba1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:13.350230 24108 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1aad3023ce0c41e982be53c5b7ad1ba1 [term 1 LEADER]: Becoming Leader. State: Replica: 1aad3023ce0c41e982be53c5b7ad1ba1, State: Running, Role: LEADER
I20260926 11:44:13.350544 24108 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1aad3023ce0c41e982be53c5b7ad1ba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aad3023ce0c41e982be53c5b7ad1ba1" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 38225 } }
I20260926 11:44:13.351094 24103 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1aad3023ce0c41e982be53c5b7ad1ba1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:13.352151 24109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aad3023ce0c41e982be53c5b7ad1ba1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1aad3023ce0c41e982be53c5b7ad1ba1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aad3023ce0c41e982be53c5b7ad1ba1" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 38225 } } }
I20260926 11:44:13.352281 24109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aad3023ce0c41e982be53c5b7ad1ba1 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:13.352622 24110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aad3023ce0c41e982be53c5b7ad1ba1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1aad3023ce0c41e982be53c5b7ad1ba1. Latest consensus state: current_term: 1 leader_uuid: "1aad3023ce0c41e982be53c5b7ad1ba1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aad3023ce0c41e982be53c5b7ad1ba1" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 38225 } } }
I20260926 11:44:13.352689 24117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:13.352715 24110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aad3023ce0c41e982be53c5b7ad1ba1 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:13.355700 24117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:13.361196 24117 catalog_manager.cc:1347] Generated new cluster ID: 04209aea9298442e8c45bea4ffb13f98
I20260926 11:44:13.361276 24117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:13.369310 24117 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:13.370560 24117 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:13.376910 24117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1aad3023ce0c41e982be53c5b7ad1ba1: Generated new TSK 0
I20260926 11:44:13.377745 24117 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:44:13.383394 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:38225
--builtin_ntp_servers=127.6.96.20:37051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 11:44:13.492704 24127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:13.492969 24127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:13.493007 24127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:13.497049 24127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:13.497113 24127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:13.497174 24127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:44:13.502004 24127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:37051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:38225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:13.503172 24127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:13.504559 24127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:13.514036 24132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:13.514127 24127 server_base.cc:1034] running on GCE node
W20260926 11:44:13.514052 24133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:13.514036 24135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:13.514698 24127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:13.515417 24127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:13.516572 24127 hybrid_clock.cc:648] HybridClock initialized: now 1790423053516548 us; error 39 us; skew 500 ppm
I20260926 11:44:13.518373 24127 webserver.cc:460] Webserver started at http://127.6.96.1:36847/ using document root <none> and password file <none>
I20260926 11:44:13.518975 24127 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:13.519061 24127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:13.519259 24127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:13.521327 24127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "3f2b9698689e48de96177e1e8964939d"
format_stamp: "Formatted at 2026-09-26 11:44:13 on dist-test-slave-xxxx"
server_key: "92aded0a428abb79c6f773aaeee465fb"
server_key_iv: "994a4d6605fa13fa4ed791a32406980a"
server_key_version: "encryptionkey@0"
I20260926 11:44:13.521864 24127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "3f2b9698689e48de96177e1e8964939d"
format_stamp: "Formatted at 2026-09-26 11:44:13 on dist-test-slave-xxxx"
server_key: "92aded0a428abb79c6f773aaeee465fb"
server_key_iv: "994a4d6605fa13fa4ed791a32406980a"
server_key_version: "encryptionkey@0"
I20260926 11:44:13.525586 24127 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260926 11:44:13.528105 24141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:13.529347 24127 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 11:44:13.529470 24127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "3f2b9698689e48de96177e1e8964939d"
format_stamp: "Formatted at 2026-09-26 11:44:13 on dist-test-slave-xxxx"
server_key: "92aded0a428abb79c6f773aaeee465fb"
server_key_iv: "994a4d6605fa13fa4ed791a32406980a"
server_key_version: "encryptionkey@0"
I20260926 11:44:13.529573 24127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:13.541712 24127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:13.542392 24127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:13.542567 24127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:13.543213 24127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:44:13.544209 24127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:44:13.544257 24127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:13.544309 24127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:44:13.544337 24127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:13.552805 24127 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:35193
I20260926 11:44:13.552827 24254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:35193 every 8 connection(s)
I20260926 11:44:13.553701 24127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:44:13.559825 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 24127
I20260926 11:44:13.559952 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:44:13.560218 6528 external_mini_cluster.cc:1428] Setting key b887c72068a09153ecdd5980c4ce4fd1
I20260926 11:44:13.562218 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:38225
--builtin_ntp_servers=127.6.96.20:37051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 11:44:13.563576 24106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:13.555467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:44807 (local address 127.6.96.62:38225)
0926 11:44:13.555614 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:13.555619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:13.556878 (+ 1259us) server_negotiation.cc:407] Connection header received
0926 11:44:13.558556 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:13.558558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:13.558616 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:13.558693 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:13.559603 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.560202 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.561609 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.561851 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.561928 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:13.563115 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:13.563119 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:13.563121 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:13.563132 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:13.563152 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:13.563160 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:13.563227 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:13.563366 (+ 139us) server_negotiation.cc:299] Negotiation successful
0926 11:44:13.563427 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 11:44:13.564472 24257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:13.556009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:38225 (local address 127.6.96.1:44807)
0926 11:44:13.556689 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:13.556744 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:13.558363 (+ 1619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:13.558880 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:13.558890 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:13.558979 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:13.559471 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:13.559484 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.560351 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.560355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:13.561507 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:13.561517 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.561995 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.561999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:13.562202 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:13.562791 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 11:44:13.562793 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:13.562866 (+ 73us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 11:44:13.562948 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:13.563237 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:13.563244 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:13.563249 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 11:44:13.563365 (+ 116us) client_negotiation.cc:240] Negotiation successful
0926 11:44:13.563666 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":128,"threads_started":1}
I20260926 11:44:13.565709 24255 heartbeater.cc:347] Connected to a master server at 127.6.96.62:38225
I20260926 11:44:13.565933 24255 heartbeater.cc:464] Registering TS with master...
I20260926 11:44:13.566465 24255 heartbeater.cc:511] Master 127.6.96.62:38225 requested a full tablet report, sending...
I20260926 11:44:13.568001 24063 ts_manager.cc:194] Registered new tserver with Master: 3f2b9698689e48de96177e1e8964939d (127.6.96.1:35193)
I20260926 11:44:13.569202 24063 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.96.1:44807
W20260926 11:44:13.671380 24258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:13.671684 24258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:13.671738 24258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:13.675854 24258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:13.675921 24258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:13.675998 24258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:44:13.681129 24258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:37051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:38225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:13.682269 24258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:13.683818 24258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:13.693683 24263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:13.693683 24266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:13.693722 24264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:13.693854 24258 server_base.cc:1034] running on GCE node
I20260926 11:44:13.694365 24258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:13.695044 24258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:13.696210 24258 hybrid_clock.cc:648] HybridClock initialized: now 1790423053696190 us; error 34 us; skew 500 ppm
I20260926 11:44:13.698154 24258 webserver.cc:460] Webserver started at http://127.6.96.2:36111/ using document root <none> and password file <none>
I20260926 11:44:13.698720 24258 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:13.698781 24258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:13.699052 24258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:13.701220 24258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "37a8c85be18a4cd59b9d11ef4eb72bb8"
format_stamp: "Formatted at 2026-09-26 11:44:13 on dist-test-slave-xxxx"
server_key: "3a74b2630179cb9b2e45b9ac8113685e"
server_key_iv: "7e03675ca3df466a8fae63f592bebf83"
server_key_version: "encryptionkey@0"
I20260926 11:44:13.701751 24258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "37a8c85be18a4cd59b9d11ef4eb72bb8"
format_stamp: "Formatted at 2026-09-26 11:44:13 on dist-test-slave-xxxx"
server_key: "3a74b2630179cb9b2e45b9ac8113685e"
server_key_iv: "7e03675ca3df466a8fae63f592bebf83"
server_key_version: "encryptionkey@0"
I20260926 11:44:13.705577 24258 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 11:44:13.708196 24272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:13.709447 24258 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 11:44:13.709546 24258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "37a8c85be18a4cd59b9d11ef4eb72bb8"
format_stamp: "Formatted at 2026-09-26 11:44:13 on dist-test-slave-xxxx"
server_key: "3a74b2630179cb9b2e45b9ac8113685e"
server_key_iv: "7e03675ca3df466a8fae63f592bebf83"
server_key_version: "encryptionkey@0"
I20260926 11:44:13.709658 24258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:13.725761 24258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:13.726440 24258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:13.726595 24258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:13.727283 24258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:44:13.728199 24258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:44:13.728245 24258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:13.728284 24258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:44:13.728304 24258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:13.736454 24258 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:35047
I20260926 11:44:13.736495 24385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:35047 every 8 connection(s)
I20260926 11:44:13.737330 24258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:44:13.738724 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 24258
I20260926 11:44:13.738880 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:44:13.739163 6528 external_mini_cluster.cc:1428] Setting key 105e98492b53e1b1046f9386ab394274
I20260926 11:44:13.741190 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:38225
--builtin_ntp_servers=127.6.96.20:37051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 11:44:13.747375 24106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:13.739646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:50643 (local address 127.6.96.62:38225)
0926 11:44:13.739781 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:13.739786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:13.740465 (+ 679us) server_negotiation.cc:407] Connection header received
0926 11:44:13.742208 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:13.742211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:13.742256 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:13.742315 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:13.743319 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.744086 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.745253 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.745601 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.745719 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:13.746808 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:13.746812 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:13.746815 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:13.746845 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:13.746871 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:13.746883 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:13.746982 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:13.747143 (+ 161us) server_negotiation.cc:299] Negotiation successful
0926 11:44:13.747216 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 11:44:13.748180 24388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:13.739611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:38225 (local address 127.6.96.2:50643)
0926 11:44:13.740334 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:13.740385 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:13.741970 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:13.742516 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:13.742526 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:13.742642 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:13.743186 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:13.743198 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.744208 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.744211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:13.745140 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:13.745149 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.745758 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.745762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:13.745915 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:13.746542 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 11:44:13.746544 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:13.746586 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 11:44:13.746650 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:13.746980 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:13.747015 (+ 35us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:13.747019 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 11:44:13.747123 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 11:44:13.747358 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":128,"threads_started":1}
I20260926 11:44:13.749445 24386 heartbeater.cc:347] Connected to a master server at 127.6.96.62:38225
I20260926 11:44:13.749701 24386 heartbeater.cc:464] Registering TS with master...
I20260926 11:44:13.750340 24386 heartbeater.cc:511] Master 127.6.96.62:38225 requested a full tablet report, sending...
I20260926 11:44:13.751477 24063 ts_manager.cc:194] Registered new tserver with Master: 37a8c85be18a4cd59b9d11ef4eb72bb8 (127.6.96.2:35047)
I20260926 11:44:13.752103 24063 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.96.2:50643
W20260926 11:44:13.852221 24389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:13.852507 24389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:13.852561 24389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:13.856771 24389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:13.856825 24389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:13.856896 24389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:44:13.861613 24389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:37051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:38225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:13.862632 24389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:13.863993 24389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:13.873080 24394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:13.873160 24395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:13.873188 24389 server_base.cc:1034] running on GCE node
W20260926 11:44:13.873418 24397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:13.873808 24389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:13.874441 24389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:13.875615 24389 hybrid_clock.cc:648] HybridClock initialized: now 1790423053875595 us; error 34 us; skew 500 ppm
I20260926 11:44:13.877503 24389 webserver.cc:460] Webserver started at http://127.6.96.3:36423/ using document root <none> and password file <none>
I20260926 11:44:13.878033 24389 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:13.878093 24389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:13.878300 24389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:13.880448 24389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "e3c2a366a41540eda99a23a23a11b293"
format_stamp: "Formatted at 2026-09-26 11:44:13 on dist-test-slave-xxxx"
server_key: "92aaa601448e40c4c47a5937324f5f40"
server_key_iv: "da275462b2497503cf0b2ca894902d48"
server_key_version: "encryptionkey@0"
I20260926 11:44:13.880992 24389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "e3c2a366a41540eda99a23a23a11b293"
format_stamp: "Formatted at 2026-09-26 11:44:13 on dist-test-slave-xxxx"
server_key: "92aaa601448e40c4c47a5937324f5f40"
server_key_iv: "da275462b2497503cf0b2ca894902d48"
server_key_version: "encryptionkey@0"
I20260926 11:44:13.884755 24389 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 11:44:13.887295 24403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:13.888510 24389 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 11:44:13.888610 24389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "e3c2a366a41540eda99a23a23a11b293"
format_stamp: "Formatted at 2026-09-26 11:44:13 on dist-test-slave-xxxx"
server_key: "92aaa601448e40c4c47a5937324f5f40"
server_key_iv: "da275462b2497503cf0b2ca894902d48"
server_key_version: "encryptionkey@0"
I20260926 11:44:13.888728 24389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:13.919600 24389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:13.920305 24389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:13.920476 24389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:13.921063 24389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:44:13.922039 24389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:44:13.922086 24389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:13.922127 24389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:44:13.922143 24389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:13.930511 24389 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:36019
I20260926 11:44:13.930622 24516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:36019 every 8 connection(s)
I20260926 11:44:13.931470 24389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:44:13.937750 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 24389
I20260926 11:44:13.937844 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:44:13.938126 6528 external_mini_cluster.cc:1428] Setting key b8808c2b6ea46aeeee50731d1865756a
I20260926 11:44:13.940707 24106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:13.933244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:42523 (local address 127.6.96.62:38225)
0926 11:44:13.933407 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:13.933411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:13.934428 (+ 1017us) server_negotiation.cc:407] Connection header received
0926 11:44:13.935833 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:13.935836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:13.935897 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:13.935980 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:13.936889 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.937442 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.938750 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.939087 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.939180 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:13.940255 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:13.940258 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:13.940261 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:13.940274 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:13.940296 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:13.940304 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:13.940359 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:13.940498 (+ 139us) server_negotiation.cc:299] Negotiation successful
0926 11:44:13.940559 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 11:44:13.941541 24519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:13.933623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:38225 (local address 127.6.96.3:42523)
0926 11:44:13.934285 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:13.934327 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:13.935619 (+ 1292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:13.936160 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:13.936175 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:13.936264 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:13.936751 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:13.936765 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.937564 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.937568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:13.938636 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:13.938646 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.939233 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.939237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:13.939394 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:13.939979 (+ 585us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 11:44:13.939981 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:13.940035 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 11:44:13.940113 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:13.940368 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:13.940377 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:13.940382 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 11:44:13.940495 (+ 113us) client_negotiation.cc:240] Negotiation successful
0926 11:44:13.940775 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":106,"threads_started":1}
I20260926 11:44:13.942579 24517 heartbeater.cc:347] Connected to a master server at 127.6.96.62:38225
I20260926 11:44:13.942811 24517 heartbeater.cc:464] Registering TS with master...
I20260926 11:44:13.943329 24517 heartbeater.cc:511] Master 127.6.96.62:38225 requested a full tablet report, sending...
I20260926 11:44:13.944319 24063 ts_manager.cc:194] Registered new tserver with Master: e3c2a366a41540eda99a23a23a11b293 (127.6.96.3:36019)
I20260926 11:44:13.944864 24063 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.96.3:42523
I20260926 11:44:13.951881 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 11:44:13.957557 24106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:13.953896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54368 (local address 127.6.96.62:38225)
0926 11:44:13.954023 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:13.954028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:13.954202 (+ 174us) server_negotiation.cc:407] Connection header received
0926 11:44:13.954317 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:13.954320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:13.954362 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:13.954428 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:13.954650 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.955229 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.956085 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.956331 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.956415 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:13.957216 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:13.957219 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:13.957221 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:13.957231 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:13.957246 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:13.957253 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:13.957299 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:13.957366 (+ 67us) server_negotiation.cc:299] Negotiation successful
0926 11:44:13.957415 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 11:44:13.960403 24063 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54368:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 11:44:13.963428 24063 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:44:13.977943 24530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:13.972276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:35047 (local address 127.0.0.1:42958)
0926 11:44:13.973153 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:13.973166 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:13.973264 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:13.973745 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:13.973752 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:13.973785 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:13.973916 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:13.973922 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.975399 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.975402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:13.976661 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:13.976671 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.977545 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.977547 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:13.977624 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:13.977655 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 11:44:13.977727 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 11:44:13.977816 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"mutex_wait_us":106,"spinlock_wait_cycles":3328,"thread_start_us":91,"threads_started":1}
I20260926 11:44:13.978060 24532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:13.972719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42958 (local address 127.6.96.2:35047)
0926 11:44:13.973050 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:13.973061 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:13.973352 (+ 291us) server_negotiation.cc:407] Connection header received
0926 11:44:13.973426 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:13.973430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:13.973600 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:13.973728 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:13.974034 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.975270 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.976780 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.977434 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.977535 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:13.977736 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:13.977788 (+ 52us) server_negotiation.cc:299] Negotiation successful
0926 11:44:13.977884 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":113,"threads_started":1}
I20260926 11:44:13.978746 24531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:13.972639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:35193 (local address 127.0.0.1:33712)
0926 11:44:13.973261 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:13.973273 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:13.973352 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:13.973761 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:13.973764 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:13.973790 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:13.973923 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:13.973929 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.975713 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.975717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:13.977085 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:13.977091 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.978390 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.978393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:13.978456 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:13.978474 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 11:44:13.978538 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 11:44:13.978607 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"mutex_wait_us":141,"spinlock_wait_cycles":1280,"thread_start_us":83,"threads_started":1}
I20260926 11:44:13.978873 24533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:13.973039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33712 (local address 127.6.96.1:35193)
0926 11:44:13.973374 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:13.973387 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:13.973410 (+ 23us) server_negotiation.cc:407] Connection header received
0926 11:44:13.973477 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:13.973480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:13.973628 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:13.973768 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:13.974055 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.975552 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.977206 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.977997 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.978097 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:13.978289 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:13.978596 (+ 307us) server_negotiation.cc:299] Negotiation successful
0926 11:44:13.978699 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":130,"threads_started":1}
I20260926 11:44:13.979595 24528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:13.971939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:36019 (local address 127.0.0.1:38908)
0926 11:44:13.972847 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:13.972900 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:13.973026 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:13.974369 (+ 1343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:13.974372 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:13.974388 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:13.974475 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:13.974480 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.975580 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.975583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:13.977076 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:13.977087 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.979238 (+ 2151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.979242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:13.979328 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:13.979351 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 11:44:13.979397 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 11:44:13.979463 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"mutex_wait_us":143,"thread_start_us":94,"threads_started":1}
I20260926 11:44:13.980223 24529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:13.972091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38908 (local address 127.6.96.3:36019)
0926 11:44:13.974039 (+ 1948us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:13.974051 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:13.974072 (+ 21us) server_negotiation.cc:407] Connection header received
0926 11:44:13.974128 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:13.974132 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:13.974259 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:13.974375 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:13.974561 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.975455 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.978348 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:13.979112 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:13.979767 (+ 655us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:13.979939 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:13.979989 (+ 50us) server_negotiation.cc:299] Negotiation successful
0926 11:44:13.980076 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1839,"thread_start_us":184,"threads_started":1}
I20260926 11:44:13.981344 24320 tablet_service.cc:1467] Processing CreateTablet for tablet 5c811b19c7a64b3683fa56024605712b (DEFAULT_TABLE table=test-table [id=536a9f7a5383483b89da7e5c59620e25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:44:13.981649 24189 tablet_service.cc:1467] Processing CreateTablet for tablet 5c811b19c7a64b3683fa56024605712b (DEFAULT_TABLE table=test-table [id=536a9f7a5383483b89da7e5c59620e25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:44:13.982329 24320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c811b19c7a64b3683fa56024605712b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:13.982589 24189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c811b19c7a64b3683fa56024605712b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:13.985682 24451 tablet_service.cc:1467] Processing CreateTablet for tablet 5c811b19c7a64b3683fa56024605712b (DEFAULT_TABLE table=test-table [id=536a9f7a5383483b89da7e5c59620e25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:44:13.986704 24451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c811b19c7a64b3683fa56024605712b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:13.988437 24534 tablet_bootstrap.cc:492] T 5c811b19c7a64b3683fa56024605712b P 37a8c85be18a4cd59b9d11ef4eb72bb8: Bootstrap starting.
I20260926 11:44:13.990083 24535 tablet_bootstrap.cc:492] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d: Bootstrap starting.
I20260926 11:44:13.991153 24534 tablet_bootstrap.cc:654] T 5c811b19c7a64b3683fa56024605712b P 37a8c85be18a4cd59b9d11ef4eb72bb8: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:13.991896 24534 log.cc:824] T 5c811b19c7a64b3683fa56024605712b P 37a8c85be18a4cd59b9d11ef4eb72bb8: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:13.992216 24535 tablet_bootstrap.cc:654] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:13.993018 24535 log.cc:824] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:13.993643 24537 tablet_bootstrap.cc:492] T 5c811b19c7a64b3683fa56024605712b P e3c2a366a41540eda99a23a23a11b293: Bootstrap starting.
I20260926 11:44:13.994396 24534 tablet_bootstrap.cc:492] T 5c811b19c7a64b3683fa56024605712b P 37a8c85be18a4cd59b9d11ef4eb72bb8: No bootstrap required, opened a new log
I20260926 11:44:13.994601 24534 ts_tablet_manager.cc:1397] T 5c811b19c7a64b3683fa56024605712b P 37a8c85be18a4cd59b9d11ef4eb72bb8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 11:44:13.994850 24535 tablet_bootstrap.cc:492] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d: No bootstrap required, opened a new log
I20260926 11:44:13.995031 24535 ts_tablet_manager.cc:1397] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260926 11:44:13.996258 24537 tablet_bootstrap.cc:654] T 5c811b19c7a64b3683fa56024605712b P e3c2a366a41540eda99a23a23a11b293: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:13.997041 24537 log.cc:824] T 5c811b19c7a64b3683fa56024605712b P e3c2a366a41540eda99a23a23a11b293: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:13.998265 24535 raft_consensus.cc:348] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c2a366a41540eda99a23a23a11b293" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36019 } } peers { permanent_uuid: "3f2b9698689e48de96177e1e8964939d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 35193 } } peers { permanent_uuid: "37a8c85be18a4cd59b9d11ef4eb72bb8" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35047 } }
I20260926 11:44:13.998499 24535 raft_consensus.cc:374] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:13.998323 24534 raft_consensus.cc:348] T 5c811b19c7a64b3683fa56024605712b P 37a8c85be18a4cd59b9d11ef4eb72bb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c2a366a41540eda99a23a23a11b293" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36019 } } peers { permanent_uuid: "3f2b9698689e48de96177e1e8964939d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 35193 } } peers { permanent_uuid: "37a8c85be18a4cd59b9d11ef4eb72bb8" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35047 } }
I20260926 11:44:13.998559 24535 raft_consensus.cc:729] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f2b9698689e48de96177e1e8964939d, State: Initialized, Role: FOLLOWER
I20260926 11:44:13.998585 24534 raft_consensus.cc:374] T 5c811b19c7a64b3683fa56024605712b P 37a8c85be18a4cd59b9d11ef4eb72bb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:13.998651 24534 raft_consensus.cc:729] T 5c811b19c7a64b3683fa56024605712b P 37a8c85be18a4cd59b9d11ef4eb72bb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37a8c85be18a4cd59b9d11ef4eb72bb8, State: Initialized, Role: FOLLOWER
I20260926 11:44:13.999058 24537 tablet_bootstrap.cc:492] T 5c811b19c7a64b3683fa56024605712b P e3c2a366a41540eda99a23a23a11b293: No bootstrap required, opened a new log
I20260926 11:44:13.999006 24535 consensus_queue.cc:260] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c2a366a41540eda99a23a23a11b293" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36019 } } peers { permanent_uuid: "3f2b9698689e48de96177e1e8964939d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 35193 } } peers { permanent_uuid: "37a8c85be18a4cd59b9d11ef4eb72bb8" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35047 } }
I20260926 11:44:13.999090 24534 consensus_queue.cc:260] T 5c811b19c7a64b3683fa56024605712b P 37a8c85be18a4cd59b9d11ef4eb72bb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c2a366a41540eda99a23a23a11b293" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36019 } } peers { permanent_uuid: "3f2b9698689e48de96177e1e8964939d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 35193 } } peers { permanent_uuid: "37a8c85be18a4cd59b9d11ef4eb72bb8" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35047 } }
I20260926 11:44:13.999231 24537 ts_tablet_manager.cc:1397] T 5c811b19c7a64b3683fa56024605712b P e3c2a366a41540eda99a23a23a11b293: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 11:44:13.999717 24386 heartbeater.cc:503] Master 127.6.96.62:38225 was elected leader, sending a full tablet report...
I20260926 11:44:13.999940 24535 ts_tablet_manager.cc:1428] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260926 11:44:14.000022 24534 ts_tablet_manager.cc:1428] T 5c811b19c7a64b3683fa56024605712b P 37a8c85be18a4cd59b9d11ef4eb72bb8: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 11:44:14.000236 24255 heartbeater.cc:503] Master 127.6.96.62:38225 was elected leader, sending a full tablet report...
I20260926 11:44:14.002904 24537 raft_consensus.cc:348] T 5c811b19c7a64b3683fa56024605712b P e3c2a366a41540eda99a23a23a11b293 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c2a366a41540eda99a23a23a11b293" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36019 } } peers { permanent_uuid: "3f2b9698689e48de96177e1e8964939d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 35193 } } peers { permanent_uuid: "37a8c85be18a4cd59b9d11ef4eb72bb8" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35047 } }
I20260926 11:44:14.003166 24537 raft_consensus.cc:374] T 5c811b19c7a64b3683fa56024605712b P e3c2a366a41540eda99a23a23a11b293 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:14.003225 24537 raft_consensus.cc:729] T 5c811b19c7a64b3683fa56024605712b P e3c2a366a41540eda99a23a23a11b293 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3c2a366a41540eda99a23a23a11b293, State: Initialized, Role: FOLLOWER
I20260926 11:44:14.003680 24537 consensus_queue.cc:260] T 5c811b19c7a64b3683fa56024605712b P e3c2a366a41540eda99a23a23a11b293 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c2a366a41540eda99a23a23a11b293" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36019 } } peers { permanent_uuid: "3f2b9698689e48de96177e1e8964939d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 35193 } } peers { permanent_uuid: "37a8c85be18a4cd59b9d11ef4eb72bb8" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35047 } }
I20260926 11:44:14.004256 24517 heartbeater.cc:503] Master 127.6.96.62:38225 was elected leader, sending a full tablet report...
I20260926 11:44:14.004648 24537 ts_tablet_manager.cc:1428] T 5c811b19c7a64b3683fa56024605712b P e3c2a366a41540eda99a23a23a11b293: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260926 11:44:14.054913 24256 tablet.cc:2369] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 11:44:14.182454 24518 tablet.cc:2369] T 5c811b19c7a64b3683fa56024605712b P e3c2a366a41540eda99a23a23a11b293: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:44:14.186616 24540 raft_consensus.cc:482] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:44:14.186872 24540 raft_consensus.cc:504] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c2a366a41540eda99a23a23a11b293" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36019 } } peers { permanent_uuid: "3f2b9698689e48de96177e1e8964939d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 35193 } } peers { permanent_uuid: "37a8c85be18a4cd59b9d11ef4eb72bb8" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35047 } }
I20260926 11:44:14.188090 24540 leader_election.cc:288] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3c2a366a41540eda99a23a23a11b293 (127.6.96.3:36019), 37a8c85be18a4cd59b9d11ef4eb72bb8 (127.6.96.2:35047)
I20260926 11:44:14.193325 24529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:14.188436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:59733 (local address 127.6.96.3:36019)
0926 11:44:14.188573 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:14.188578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:14.188784 (+ 206us) server_negotiation.cc:407] Connection header received
0926 11:44:14.188941 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:14.188944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:14.188995 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:14.189077 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:14.189447 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:14.190244 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:14.192001 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:14.192643 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:14.192753 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:14.192817 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:14.193110 (+ 293us) server_negotiation.cc:299] Negotiation successful
0926 11:44:14.193188 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 11:44:14.193325 24532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:14.188491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:50621 (local address 127.6.96.2:35047)
0926 11:44:14.188615 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:14.188619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:14.188801 (+ 182us) server_negotiation.cc:407] Connection header received
0926 11:44:14.188940 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:14.188943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:14.188995 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:14.189091 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:14.189447 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:14.190244 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:14.192001 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:14.192643 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:14.192753 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:14.192817 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:14.193110 (+ 293us) server_negotiation.cc:299] Negotiation successful
0926 11:44:14.193188 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 11:44:14.193377 24544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:14.188401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:35047 (local address 127.6.96.1:50621)
0926 11:44:14.188729 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:14.188740 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:14.188838 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:14.189095 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:14.189098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:14.189122 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:14.189344 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:14.189351 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:14.190355 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:14.190359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:14.191845 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:14.191860 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:14.192766 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:14.192770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:14.192965 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:14.193008 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 11:44:14.193084 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 11:44:14.193194 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":179,"spinlock_wait_cycles":11904,"thread_start_us":76,"threads_started":1}
I20260926 11:44:14.193375 24543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:14.188327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:36019 (local address 127.6.96.1:59733)
0926 11:44:14.188682 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:14.188697 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:14.188838 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:14.189095 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:14.189098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:14.189122 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:14.189348 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:14.189353 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:14.190356 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:14.190359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:14.191845 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:14.191860 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:14.192766 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:14.192770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:14.192969 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:14.193013 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 11:44:14.193084 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 11:44:14.193213 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":193,"spinlock_wait_cycles":21888,"thread_start_us":99,"threads_started":1}
I20260926 11:44:14.194068 24340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c811b19c7a64b3683fa56024605712b" candidate_uuid: "3f2b9698689e48de96177e1e8964939d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37a8c85be18a4cd59b9d11ef4eb72bb8" is_pre_election: true
I20260926 11:44:14.194114 24471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c811b19c7a64b3683fa56024605712b" candidate_uuid: "3f2b9698689e48de96177e1e8964939d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3c2a366a41540eda99a23a23a11b293" is_pre_election: true
I20260926 11:44:14.194379 24340 raft_consensus.cc:2454] T 5c811b19c7a64b3683fa56024605712b P 37a8c85be18a4cd59b9d11ef4eb72bb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f2b9698689e48de96177e1e8964939d in term 0.
I20260926 11:44:14.194370 24471 raft_consensus.cc:2454] T 5c811b19c7a64b3683fa56024605712b P e3c2a366a41540eda99a23a23a11b293 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f2b9698689e48de96177e1e8964939d in term 0.
I20260926 11:44:14.194880 24142 leader_election.cc:302] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37a8c85be18a4cd59b9d11ef4eb72bb8, 3f2b9698689e48de96177e1e8964939d; no voters:
I20260926 11:44:14.195164 24540 raft_consensus.cc:2781] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:44:14.195258 24540 raft_consensus.cc:482] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:44:14.195292 24540 raft_consensus.cc:3037] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:14.196316 24540 raft_consensus.cc:504] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c2a366a41540eda99a23a23a11b293" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36019 } } peers { permanent_uuid: "3f2b9698689e48de96177e1e8964939d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 35193 } } peers { permanent_uuid: "37a8c85be18a4cd59b9d11ef4eb72bb8" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35047 } }
I20260926 11:44:14.196694 24540 leader_election.cc:288] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d [CANDIDATE]: Term 1 election: Requested vote from peers e3c2a366a41540eda99a23a23a11b293 (127.6.96.3:36019), 37a8c85be18a4cd59b9d11ef4eb72bb8 (127.6.96.2:35047)
I20260926 11:44:14.197029 24471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c811b19c7a64b3683fa56024605712b" candidate_uuid: "3f2b9698689e48de96177e1e8964939d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3c2a366a41540eda99a23a23a11b293"
I20260926 11:44:14.197166 24471 raft_consensus.cc:3037] T 5c811b19c7a64b3683fa56024605712b P e3c2a366a41540eda99a23a23a11b293 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:14.197172 24340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c811b19c7a64b3683fa56024605712b" candidate_uuid: "3f2b9698689e48de96177e1e8964939d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37a8c85be18a4cd59b9d11ef4eb72bb8"
I20260926 11:44:14.197294 24340 raft_consensus.cc:3037] T 5c811b19c7a64b3683fa56024605712b P 37a8c85be18a4cd59b9d11ef4eb72bb8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:14.198210 24471 raft_consensus.cc:2454] T 5c811b19c7a64b3683fa56024605712b P e3c2a366a41540eda99a23a23a11b293 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f2b9698689e48de96177e1e8964939d in term 1.
I20260926 11:44:14.198268 24340 raft_consensus.cc:2454] T 5c811b19c7a64b3683fa56024605712b P 37a8c85be18a4cd59b9d11ef4eb72bb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f2b9698689e48de96177e1e8964939d in term 1.
I20260926 11:44:14.198554 24143 leader_election.cc:302] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f2b9698689e48de96177e1e8964939d, e3c2a366a41540eda99a23a23a11b293; no voters:
I20260926 11:44:14.198734 24540 raft_consensus.cc:2781] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:14.198992 24540 raft_consensus.cc:686] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d [term 1 LEADER]: Becoming Leader. State: Replica: 3f2b9698689e48de96177e1e8964939d, State: Running, Role: LEADER
I20260926 11:44:14.199306 24540 consensus_queue.cc:237] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c2a366a41540eda99a23a23a11b293" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36019 } } peers { permanent_uuid: "3f2b9698689e48de96177e1e8964939d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 35193 } } peers { permanent_uuid: "37a8c85be18a4cd59b9d11ef4eb72bb8" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35047 } }
I20260926 11:44:14.202575 24062 catalog_manager.cc:5495] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f2b9698689e48de96177e1e8964939d (127.6.96.1). New cstate: current_term: 1 leader_uuid: "3f2b9698689e48de96177e1e8964939d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c2a366a41540eda99a23a23a11b293" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f2b9698689e48de96177e1e8964939d" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 35193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37a8c85be18a4cd59b9d11ef4eb72bb8" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35047 } health_report { overall_health: UNKNOWN } } }
I20260926 11:44:14.218145 24533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:14.214811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33722 (local address 127.6.96.1:35193)
0926 11:44:14.214940 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:14.214943 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:14.214957 (+ 14us) server_negotiation.cc:407] Connection header received
0926 11:44:14.215073 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:14.215075 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:14.215133 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:14.215231 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:44:14.215499 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:14.215994 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:14.216784 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:14.216986 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:14.217057 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:14.217361 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:44:14.217786 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:44:14.217878 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:14.217980 (+ 102us) server_negotiation.cc:299] Negotiation successful
0926 11:44:14.218032 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 11:44:14.224550 24471 raft_consensus.cc:1261] T 5c811b19c7a64b3683fa56024605712b P e3c2a366a41540eda99a23a23a11b293 [term 1 FOLLOWER]: Refusing update from remote peer 3f2b9698689e48de96177e1e8964939d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:44:14.224550 24340 raft_consensus.cc:1261] T 5c811b19c7a64b3683fa56024605712b P 37a8c85be18a4cd59b9d11ef4eb72bb8 [term 1 FOLLOWER]: Refusing update from remote peer 3f2b9698689e48de96177e1e8964939d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:44:14.225328 24540 consensus_queue.cc:1035] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3c2a366a41540eda99a23a23a11b293" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:44:14.225567 24545 consensus_queue.cc:1035] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d [LEADER]: Connected to new peer: Peer: permanent_uuid: "37a8c85be18a4cd59b9d11ef4eb72bb8" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 35047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:44:14.235236 24550 mvcc.cc:205] Tried to move back new op lower bound from 7333572830095294464 to 7333572830002868224. Current Snapshot: MvccSnapshot[applied={T|T < 7333572830095294464}]
I20260926 11:44:14.242127 24062 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:44:14.242281 24062 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:44:14.244573 24062 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1aad3023ce0c41e982be53c5b7ad1ba1: Sending DeleteTablet for 3 replicas of tablet 5c811b19c7a64b3683fa56024605712b
I20260926 11:44:14.245369 24320 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c811b19c7a64b3683fa56024605712b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:44:14 UTC) from {username='slave'} at 127.0.0.1:42958
I20260926 11:44:14.245440 24451 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c811b19c7a64b3683fa56024605712b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:44:14 UTC) from {username='slave'} at 127.0.0.1:38908
I20260926 11:44:14.245872 24557 tablet_replica.cc:330] T 5c811b19c7a64b3683fa56024605712b P e3c2a366a41540eda99a23a23a11b293: stopping tablet replica
I20260926 11:44:14.245891 24189 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c811b19c7a64b3683fa56024605712b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:44:14 UTC) from {username='slave'} at 127.0.0.1:33712
I20260926 11:44:14.246196 24557 raft_consensus.cc:2229] T 5c811b19c7a64b3683fa56024605712b P e3c2a366a41540eda99a23a23a11b293 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:44:14.246318 24559 tablet_replica.cc:330] T 5c811b19c7a64b3683fa56024605712b P 3f2b9698689e48de96177e1e8964939d: stopping tablet replica
I20260926 11:44:14.246428 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 24127
I20260926 11:44:14.247314 24557 raft_consensus.cc:2258] T 5c811b19c7a64b3683fa56024605712b P e3c2a366a41540eda99a23a23a11b293 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:44:14.248147 24558 tablet_replica.cc:330] T 5c811b19c7a64b3683fa56024605712b P 37a8c85be18a4cd59b9d11ef4eb72bb8: stopping tablet replica
I20260926 11:44:14.248402 24558 raft_consensus.cc:2229] T 5c811b19c7a64b3683fa56024605712b P 37a8c85be18a4cd59b9d11ef4eb72bb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:44:14.248665 24558 raft_consensus.cc:2258] T 5c811b19c7a64b3683fa56024605712b P 37a8c85be18a4cd59b9d11ef4eb72bb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:44:14.248976 24557 ts_tablet_manager.cc:1905] T 5c811b19c7a64b3683fa56024605712b P e3c2a366a41540eda99a23a23a11b293: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:44:14.249773 24558 ts_tablet_manager.cc:1905] T 5c811b19c7a64b3683fa56024605712b P 37a8c85be18a4cd59b9d11ef4eb72bb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:44:14.251703 24557 ts_tablet_manager.cc:1918] T 5c811b19c7a64b3683fa56024605712b P e3c2a366a41540eda99a23a23a11b293: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:44:14.251832 24557 log.cc:1196] T 5c811b19c7a64b3683fa56024605712b P e3c2a366a41540eda99a23a23a11b293: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/5c811b19c7a64b3683fa56024605712b
I20260926 11:44:14.251924 24558 ts_tablet_manager.cc:1918] T 5c811b19c7a64b3683fa56024605712b P 37a8c85be18a4cd59b9d11ef4eb72bb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:44:14.252017 24558 log.cc:1196] T 5c811b19c7a64b3683fa56024605712b P 37a8c85be18a4cd59b9d11ef4eb72bb8: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790422817500949-6528-0/minicluster-data/ts-1/wal/wals/5c811b19c7a64b3683fa56024605712b
I20260926 11:44:14.252120 24557 ts_tablet_manager.cc:1939] T 5c811b19c7a64b3683fa56024605712b P e3c2a366a41540eda99a23a23a11b293: Deleting consensus metadata
I20260926 11:44:14.252360 24558 ts_tablet_manager.cc:1939] T 5c811b19c7a64b3683fa56024605712b P 37a8c85be18a4cd59b9d11ef4eb72bb8: Deleting consensus metadata
I20260926 11:44:14.253364 24052 catalog_manager.cc:4841] TS e3c2a366a41540eda99a23a23a11b293 (127.6.96.3:36019): tablet 5c811b19c7a64b3683fa56024605712b (table test-table [id=536a9f7a5383483b89da7e5c59620e25]) successfully deleted
I20260926 11:44:14.253528 24051 catalog_manager.cc:4841] TS 37a8c85be18a4cd59b9d11ef4eb72bb8 (127.6.96.2:35047): tablet 5c811b19c7a64b3683fa56024605712b (table test-table [id=536a9f7a5383483b89da7e5c59620e25]) successfully deleted
W20260926 11:44:14.253952 24052 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.96.1:35193 (error 108)
I20260926 11:44:14.254081 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 24258
I20260926 11:44:14.255352 24531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:14.255093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:35193 (local address 127.0.0.1:33730)
0926 11:44:14.255207 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:14.255272 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.1:35193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260926 11:44:14.255782 24052 catalog_manager.cc:4584] TS 3f2b9698689e48de96177e1e8964939d (127.6.96.1:35193): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5c811b19c7a64b3683fa56024605712b: Network error: Client connection negotiation failed: client connection to 127.6.96.1:35193: connect: Connection refused (error 111)
I20260926 11:44:14.259341 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 24389
I20260926 11:44:14.263667 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 24036
2026-09-26T11:44:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1132 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:44:14 dist-test-slave-xxxx krb5kdc[24563](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:44:14 dist-test-slave-xxxx krb5kdc[24563](info): set up 2 sockets
Sep 26 11:44:14 dist-test-slave-xxxx krb5kdc[24563](info): commencing operation
krb5kdc: starting...
W20260926 11:44:15.214536 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:44:15 dist-test-slave-xxxx krb5kdc[24563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:44:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:44:15.334488 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:34925
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:37049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:34925
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:15.442453 24576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:15.442710 24576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:15.442749 24576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:15.446719 24576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:15.446786 24576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:15.446805 24576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:15.446820 24576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:15.446853 24576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:15.452004 24576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:37049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:34925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:34925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:15.452955 24576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:15.453868 24576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:15.461746 24581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:15.461831 24576 server_base.cc:1034] running on GCE node
W20260926 11:44:15.461746 24582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:15.461750 24584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:15.462534 24576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:15.463644 24576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:15.464805 24576 hybrid_clock.cc:648] HybridClock initialized: now 1790423055464792 us; error 29 us; skew 500 ppm
Sep 26 11:44:15 dist-test-slave-xxxx krb5kdc[24563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423055, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:15.467458 24576 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:44:15.468343 24576 webserver.cc:460] Webserver started at http://127.6.96.62:44627/ using document root <none> and password file <none>
I20260926 11:44:15.468856 24576 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:15.468910 24576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:15.469069 24576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:15.471101 24576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "16f9135b6ce6467bb9422f81b4503543"
format_stamp: "Formatted at 2026-09-26 11:44:15 on dist-test-slave-xxxx"
server_key: "7a0e725b9dee576644e37ee3ced283fb"
server_key_iv: "a504853c935b85f206b6d7b2d872f744"
server_key_version: "encryptionkey@0"
I20260926 11:44:15.471635 24576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "16f9135b6ce6467bb9422f81b4503543"
format_stamp: "Formatted at 2026-09-26 11:44:15 on dist-test-slave-xxxx"
server_key: "7a0e725b9dee576644e37ee3ced283fb"
server_key_iv: "a504853c935b85f206b6d7b2d872f744"
server_key_version: "encryptionkey@0"
I20260926 11:44:15.475298 24576 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260926 11:44:15.477727 24591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:15.478991 24576 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:44:15.479094 24576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "16f9135b6ce6467bb9422f81b4503543"
format_stamp: "Formatted at 2026-09-26 11:44:15 on dist-test-slave-xxxx"
server_key: "7a0e725b9dee576644e37ee3ced283fb"
server_key_iv: "a504853c935b85f206b6d7b2d872f744"
server_key_version: "encryptionkey@0"
I20260926 11:44:15.479200 24576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:15.504302 24576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:15.507284 24576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:15.507467 24576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:15.514092 24576 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:34925
I20260926 11:44:15.514122 24643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:34925 every 8 connection(s)
I20260926 11:44:15.515067 24576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:15.517742 24644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:15.520491 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 24576
I20260926 11:44:15.520632 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:15.520907 6528 external_mini_cluster.cc:1428] Setting key 50245871b7c47d4c6ec954c9e4f8a9d1
I20260926 11:44:15.523779 24644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543: Bootstrap starting.
I20260926 11:44:15.526428 24644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:15.527216 24644 log.cc:824] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:15.529209 24644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543: No bootstrap required, opened a new log
Sep 26 11:44:15 dist-test-slave-xxxx krb5kdc[24563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:44:15.532837 24644 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f9135b6ce6467bb9422f81b4503543" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 34925 } }
I20260926 11:44:15.533115 24644 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:15.533172 24644 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f9135b6ce6467bb9422f81b4503543, State: Initialized, Role: FOLLOWER
I20260926 11:44:15.533646 24644 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f9135b6ce6467bb9422f81b4503543" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 34925 } }
I20260926 11:44:15.533803 24644 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:44:15.533866 24644 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:44:15.533962 24644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:15.535091 24644 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f9135b6ce6467bb9422f81b4503543" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 34925 } }
I20260926 11:44:15.535462 24644 leader_election.cc:302] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16f9135b6ce6467bb9422f81b4503543; no voters:
I20260926 11:44:15.535799 24644 leader_election.cc:288] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:44:15.535940 24649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:15.536142 24649 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543 [term 1 LEADER]: Becoming Leader. State: Replica: 16f9135b6ce6467bb9422f81b4503543, State: Running, Role: LEADER
I20260926 11:44:15.536218 24647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:15.522303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40918 (local address 127.6.96.62:34925)
0926 11:44:15.522749 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:15.522767 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:15.522808 (+ 41us) server_negotiation.cc:407] Connection header received
0926 11:44:15.523716 (+ 908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:15.523738 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:15.524164 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:15.524524 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:15.525133 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:15.526579 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:15.527500 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:15.528070 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:15.528218 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:15.530497 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:15.530503 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:15.530519 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:15.530550 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:15.533074 (+ 2524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:15.533507 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:15.533512 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:15.533518 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:15.533653 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:15.533902 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:15.533905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:15.533907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:15.534408 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:15.534611 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:15.534893 (+ 282us) server_negotiation.cc:299] Negotiation successful
0926 11:44:15.535200 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":149,"threads_started":1}
I20260926 11:44:15.536468 24649 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f9135b6ce6467bb9422f81b4503543" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 34925 } }
I20260926 11:44:15.537002 24644 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:15.539746 24650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16f9135b6ce6467bb9422f81b4503543. Latest consensus state: current_term: 1 leader_uuid: "16f9135b6ce6467bb9422f81b4503543" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f9135b6ce6467bb9422f81b4503543" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 34925 } } }
I20260926 11:44:15.539754 24651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16f9135b6ce6467bb9422f81b4503543" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f9135b6ce6467bb9422f81b4503543" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 34925 } } }
I20260926 11:44:15.539881 24650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:15.539887 24651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543 [sys.catalog]: This master's current role is: LEADER
W20260926 11:44:15.541617 24663 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 11:44:15.541678 24663 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 11:44:15.541746 24665 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:15.542418 24665 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:15.547287 24665 catalog_manager.cc:1347] Generated new cluster ID: d19060e48a984deaaf8e4b785f8c66ed
I20260926 11:44:15.547353 24665 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:15.567548 24665 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:15.568848 24665 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:15.577040 24665 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543: Generated new TSK 0
I20260926 11:44:15.577862 24665 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.96.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
I20260926 11:44:15.607877 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:34925
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:37049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:15.716149 24670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:15.716409 24670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:15.716467 24670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:15.720575 24670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:15.720634 24670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:15.720715 24670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:44:15.725883 24670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:37049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:34925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:15.726953 24670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:15.727941 24670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:15.737380 24678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:15.737327 24675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:15.737345 24676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:15.737393 24670 server_base.cc:1034] running on GCE node
I20260926 11:44:15.737915 24670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:15.738569 24670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:15.739727 24670 hybrid_clock.cc:648] HybridClock initialized: now 1790423055739687 us; error 56 us; skew 500 ppm
Sep 26 11:44:15 dist-test-slave-xxxx krb5kdc[24563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423055, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:15.742419 24670 init.cc:377] Logged in from keytab as kudu/127.6.96.1@KRBTEST.COM (short username kudu)
I20260926 11:44:15.743340 24670 webserver.cc:460] Webserver started at http://127.6.96.1:43643/ using document root <none> and password file <none>
I20260926 11:44:15.743857 24670 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:15.743914 24670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:15.744078 24670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:15.746059 24670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "38d076b82e8949edbe0c813912533768"
format_stamp: "Formatted at 2026-09-26 11:44:15 on dist-test-slave-xxxx"
server_key: "6a72d3b4b558fe1a408c8f0862921251"
server_key_iv: "b82bbef69b1e3df04dc7cf13d1d14800"
server_key_version: "encryptionkey@0"
I20260926 11:44:15.746567 24670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "38d076b82e8949edbe0c813912533768"
format_stamp: "Formatted at 2026-09-26 11:44:15 on dist-test-slave-xxxx"
server_key: "6a72d3b4b558fe1a408c8f0862921251"
server_key_iv: "b82bbef69b1e3df04dc7cf13d1d14800"
server_key_version: "encryptionkey@0"
I20260926 11:44:15.750149 24670 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 11:44:15.752523 24685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:15.753729 24670 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 11:44:15.753834 24670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "38d076b82e8949edbe0c813912533768"
format_stamp: "Formatted at 2026-09-26 11:44:15 on dist-test-slave-xxxx"
server_key: "6a72d3b4b558fe1a408c8f0862921251"
server_key_iv: "b82bbef69b1e3df04dc7cf13d1d14800"
server_key_version: "encryptionkey@0"
I20260926 11:44:15.753928 24670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:15.772634 24670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:15.775765 24670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:15.775938 24670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:15.776530 24670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:44:15.777340 24670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:44:15.777385 24670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:15.777423 24670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:44:15.777449 24670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:15.786355 24670 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:42347
I20260926 11:44:15.786384 24798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:42347 every 8 connection(s)
I20260926 11:44:15.787321 24670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:44:15.793898 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 24670
I20260926 11:44:15.794008 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:44:15.794261 6528 external_mini_cluster.cc:1428] Setting key 4058f99e9f72d4306aa6a52248b8387b
Sep 26 11:44:15 dist-test-slave-xxxx krb5kdc[24563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423055, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:44:15.800282 24647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:15.789016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:44033 (local address 127.6.96.62:34925)
0926 11:44:15.789151 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:15.789155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:15.790028 (+ 873us) server_negotiation.cc:407] Connection header received
0926 11:44:15.790899 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:15.790902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:15.790957 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:15.791034 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:15.792227 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:15.792755 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:15.793925 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:15.794243 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:15.794362 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:15.796893 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:15.796898 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:15.796901 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:15.796926 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:15.798178 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:15.798722 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:15.798726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:15.798727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:15.798808 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:15.799160 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:15.799163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:15.799164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:15.799337 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:15.799410 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:15.800037 (+ 627us) server_negotiation.cc:299] Negotiation successful
0926 11:44:15.800159 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 11:44:15.801461 24801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:15.789424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:34925 (local address 127.6.96.1:44033)
0926 11:44:15.789870 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:15.789913 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:15.790697 (+ 784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:15.791209 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:15.791216 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:15.791572 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:15.792078 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:15.792092 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:15.792871 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:15.792875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:15.793762 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:15.793772 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:15.794425 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:15.794430 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:15.794640 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:15.795264 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:15.795281 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:15.796733 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:15.798284 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:15.798291 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:15.798308 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:15.798612 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:15.798916 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:15.798919 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:15.798921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:15.799087 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:15.799429 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:15.799435 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:15.799770 (+ 335us) client_negotiation.cc:769] Sending connection context
0926 11:44:15.800002 (+ 232us) client_negotiation.cc:240] Negotiation successful
0926 11:44:15.800360 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":111,"threads_started":1}
I20260926 11:44:15.802670 24799 heartbeater.cc:347] Connected to a master server at 127.6.96.62:34925
I20260926 11:44:15.802968 24799 heartbeater.cc:464] Registering TS with master...
I20260926 11:44:15.803571 24799 heartbeater.cc:511] Master 127.6.96.62:34925 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.96.2@KRBTEST.COM; defaulting to no policy
I20260926 11:44:15.805164 24608 ts_manager.cc:194] Registered new tserver with Master: 38d076b82e8949edbe0c813912533768 (127.6.96.1:42347)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.2@KRBTEST.COM" created.
I20260926 11:44:15.806331 24608 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.1@KRBTEST.COM'} at 127.6.96.1:44033
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
I20260926 11:44:15.821750 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:34925
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:37049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:15.929430 24804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:15.929703 24804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:15.929746 24804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:15.933817 24804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:15.933874 24804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:15.933931 24804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:44:15.939268 24804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:37049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:34925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:15.940268 24804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:15.941231 24804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:15.950261 24810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:15.950237 24812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:15.950273 24809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:15.950392 24804 server_base.cc:1034] running on GCE node
I20260926 11:44:15.950982 24804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:15.951581 24804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:15.952730 24804 hybrid_clock.cc:648] HybridClock initialized: now 1790423055952711 us; error 29 us; skew 500 ppm
Sep 26 11:44:15 dist-test-slave-xxxx krb5kdc[24563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423055, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:15.955843 24804 init.cc:377] Logged in from keytab as kudu/127.6.96.2@KRBTEST.COM (short username kudu)
I20260926 11:44:15.956774 24804 webserver.cc:460] Webserver started at http://127.6.96.2:36955/ using document root <none> and password file <none>
I20260926 11:44:15.957300 24804 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:15.957360 24804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:15.957551 24804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:15.959635 24804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "488691a83e00473daa5a8f2adb837c4d"
format_stamp: "Formatted at 2026-09-26 11:44:15 on dist-test-slave-xxxx"
server_key: "f454976a22f10cb6977d5deaa1c03e0f"
server_key_iv: "8a0a40b0f9a5932dbb244065fdfa7a85"
server_key_version: "encryptionkey@0"
I20260926 11:44:15.960172 24804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "488691a83e00473daa5a8f2adb837c4d"
format_stamp: "Formatted at 2026-09-26 11:44:15 on dist-test-slave-xxxx"
server_key: "f454976a22f10cb6977d5deaa1c03e0f"
server_key_iv: "8a0a40b0f9a5932dbb244065fdfa7a85"
server_key_version: "encryptionkey@0"
I20260926 11:44:15.963908 24804 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 11:44:15.966285 24819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:15.967576 24804 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:44:15.967700 24804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "488691a83e00473daa5a8f2adb837c4d"
format_stamp: "Formatted at 2026-09-26 11:44:15 on dist-test-slave-xxxx"
server_key: "f454976a22f10cb6977d5deaa1c03e0f"
server_key_iv: "8a0a40b0f9a5932dbb244065fdfa7a85"
server_key_version: "encryptionkey@0"
I20260926 11:44:15.967800 24804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:15.990500 24804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:15.993364 24804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:15.993531 24804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:15.994087 24804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:44:15.994918 24804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:44:15.994966 24804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:15.995018 24804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:44:15.995046 24804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:16.003187 24804 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:39485
I20260926 11:44:16.003213 24932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:39485 every 8 connection(s)
I20260926 11:44:16.004032 24804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:44:16.007562 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 24804
I20260926 11:44:16.007694 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:44:16.007953 6528 external_mini_cluster.cc:1428] Setting key de7ebd4008db269cbd5777c08bea1425
Sep 26 11:44:16 dist-test-slave-xxxx krb5kdc[24563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423055, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.96.3@KRBTEST.COM; defaulting to no policy
I20260926 11:44:16.017108 24647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:16.005777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:50453 (local address 127.6.96.62:34925)
0926 11:44:16.005935 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:16.005940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:16.006799 (+ 859us) server_negotiation.cc:407] Connection header received
0926 11:44:16.007702 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:16.007706 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:16.007763 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:16.007841 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:16.009088 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.009623 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.010738 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.011002 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.011093 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:16.013560 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:16.013565 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:16.013568 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:16.013597 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:16.015147 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:16.015597 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:16.015600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:16.015602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:16.015692 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:16.015986 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:16.015990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:16.015991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:16.016224 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:16.016329 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:16.016795 (+ 466us) server_negotiation.cc:299] Negotiation successful
0926 11:44:16.016936 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 11:44:16.017867 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:16.006201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:34925 (local address 127.6.96.2:50453)
0926 11:44:16.006653 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:16.006693 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:16.007489 (+ 796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:16.008042 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:16.008051 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:16.008477 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:16.008953 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:16.008967 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.009735 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.009738 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:16.010622 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:16.010631 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.011139 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.011143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:16.011301 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:16.011813 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:16.011831 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:16.013399 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:16.015252 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:16.015256 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:16.015268 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:16.015503 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:16.015776 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:16.015778 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:16.015779 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:16.015899 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:16.016316 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:16.016320 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:16.016558 (+ 238us) client_negotiation.cc:769] Sending connection context
0926 11:44:16.016714 (+ 156us) client_negotiation.cc:240] Negotiation successful
0926 11:44:16.016972 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":107,"threads_started":1}
I20260926 11:44:16.019249 24933 heartbeater.cc:347] Connected to a master server at 127.6.96.62:34925
I20260926 11:44:16.019510 24933 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.3@KRBTEST.COM" created.
I20260926 11:44:16.020051 24933 heartbeater.cc:511] Master 127.6.96.62:34925 requested a full tablet report, sending...
I20260926 11:44:16.021250 24608 ts_manager.cc:194] Registered new tserver with Master: 488691a83e00473daa5a8f2adb837c4d (127.6.96.2:39485)
I20260926 11:44:16.021821 24608 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.2@KRBTEST.COM'} at 127.6.96.2:50453
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
I20260926 11:44:16.033862 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:34925
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:37049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:16.143307 24938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:16.143572 24938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:16.143615 24938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:16.147610 24938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:16.147665 24938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:16.147720 24938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:44:16.152951 24938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:37049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:34925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:16.153956 24938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:16.154970 24938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:16.164489 24944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:16.164495 24943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:16.164582 24938 server_base.cc:1034] running on GCE node
W20260926 11:44:16.164495 24946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:16.165148 24938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:16.165761 24938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:16.166950 24938 hybrid_clock.cc:648] HybridClock initialized: now 1790423056166928 us; error 31 us; skew 500 ppm
Sep 26 11:44:16 dist-test-slave-xxxx krb5kdc[24563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423056, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:16.169488 24938 init.cc:377] Logged in from keytab as kudu/127.6.96.3@KRBTEST.COM (short username kudu)
I20260926 11:44:16.170296 24938 webserver.cc:460] Webserver started at http://127.6.96.3:40591/ using document root <none> and password file <none>
I20260926 11:44:16.170804 24938 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:16.170941 24938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:16.171145 24938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:16.173208 24938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "3de99b0ab95b426c892a9e24c3e4f6dc"
format_stamp: "Formatted at 2026-09-26 11:44:16 on dist-test-slave-xxxx"
server_key: "969b4cb1eb1865016fc1e76ec18f243c"
server_key_iv: "b55c0e2b62c659a72de5d31eb15b81c1"
server_key_version: "encryptionkey@0"
I20260926 11:44:16.173733 24938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "3de99b0ab95b426c892a9e24c3e4f6dc"
format_stamp: "Formatted at 2026-09-26 11:44:16 on dist-test-slave-xxxx"
server_key: "969b4cb1eb1865016fc1e76ec18f243c"
server_key_iv: "b55c0e2b62c659a72de5d31eb15b81c1"
server_key_version: "encryptionkey@0"
I20260926 11:44:16.177263 24938 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260926 11:44:16.179636 24953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:16.180796 24938 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:44:16.180897 24938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "3de99b0ab95b426c892a9e24c3e4f6dc"
format_stamp: "Formatted at 2026-09-26 11:44:16 on dist-test-slave-xxxx"
server_key: "969b4cb1eb1865016fc1e76ec18f243c"
server_key_iv: "b55c0e2b62c659a72de5d31eb15b81c1"
server_key_version: "encryptionkey@0"
I20260926 11:44:16.180996 24938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:16.194039 24938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:16.196991 24938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:16.197155 24938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:16.197744 24938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:44:16.198567 24938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:44:16.198612 24938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:16.198655 24938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:44:16.198683 24938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:16.206789 24938 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:45043
I20260926 11:44:16.206818 25066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:45043 every 8 connection(s)
I20260926 11:44:16.207691 24938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:44:16.209775 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 24938
I20260926 11:44:16.209899 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:44:16.210201 6528 external_mini_cluster.cc:1428] Setting key bcb1669bc1324f2b45ebcd44eba50e16
Sep 26 11:44:16 dist-test-slave-xxxx krb5kdc[24563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423056, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:44:16.220952 24647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:16.209412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:47697 (local address 127.6.96.62:34925)
0926 11:44:16.209550 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:16.209554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:16.210652 (+ 1098us) server_negotiation.cc:407] Connection header received
0926 11:44:16.211801 (+ 1149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:16.211804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:16.211857 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:16.211927 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:16.213143 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.213668 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.214796 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.215055 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.215143 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:16.217575 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:16.217578 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:16.217580 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:16.217606 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:16.219001 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:16.219483 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:16.219487 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:16.219489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:16.219564 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:16.219866 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:16.219869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:16.219871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:16.220059 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:16.220141 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:16.220657 (+ 516us) server_negotiation.cc:299] Negotiation successful
0926 11:44:16.220788 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 11:44:16.221750 25069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:16.209871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:34925 (local address 127.6.96.3:47697)
0926 11:44:16.210438 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:16.210517 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:16.211602 (+ 1085us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:16.212099 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:16.212107 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:16.212529 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:16.213013 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:16.213026 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.213783 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.213786 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:16.214683 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:16.214692 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.215193 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.215197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:16.215372 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:16.215938 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:16.215956 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:16.217398 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:16.219117 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:16.219124 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:16.219140 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:16.219391 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:16.219641 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:16.219644 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:16.219646 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:16.219782 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:16.220155 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:16.220162 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:16.220424 (+ 262us) client_negotiation.cc:769] Sending connection context
0926 11:44:16.220595 (+ 171us) client_negotiation.cc:240] Negotiation successful
0926 11:44:16.220938 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":145,"threads_started":1}
I20260926 11:44:16.223006 25067 heartbeater.cc:347] Connected to a master server at 127.6.96.62:34925
I20260926 11:44:16.223264 25067 heartbeater.cc:464] Registering TS with master...
I20260926 11:44:16.223783 25067 heartbeater.cc:511] Master 127.6.96.62:34925 requested a full tablet report, sending...
I20260926 11:44:16.224745 24608 ts_manager.cc:194] Registered new tserver with Master: 3de99b0ab95b426c892a9e24c3e4f6dc (127.6.96.3:45043)
I20260926 11:44:16.225301 24608 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.3@KRBTEST.COM'} at 127.6.96.3:47697
I20260926 11:44:16.235509 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 11:44:16.244463 24647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:16.237341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33904 (local address 127.6.96.62:34925)
0926 11:44:16.237465 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:16.237470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:16.237612 (+ 142us) server_negotiation.cc:407] Connection header received
0926 11:44:16.237715 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:16.237717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:16.237754 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:16.237813 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:16.238480 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.239053 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.239853 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.240056 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.240124 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:16.241571 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:16.241574 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:16.241576 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:16.241600 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:16.242736 (+ 1136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:16.243256 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:16.243259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:16.243260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:16.243336 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:16.243650 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:16.243653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:16.243654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:16.243868 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:16.243993 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:16.244221 (+ 228us) server_negotiation.cc:299] Negotiation successful
0926 11:44:16.244299 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 11:44:16.247191 24608 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33904:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 11:44:16.250144 24608 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:44:16.264284 25078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:16.258523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:42347 (local address 127.0.0.1:48974)
0926 11:44:16.258881 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:16.258928 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:16.259192 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:16.259989 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:16.259995 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:16.260018 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:16.260156 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:16.260164 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.261397 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.261400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:16.262514 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:16.262524 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.263773 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.263777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:16.263880 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:16.263917 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 11:44:16.264019 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 11:44:16.264137 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":74,"threads_started":1}
I20260926 11:44:16.264441 25083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:16.259691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34176 (local address 127.6.96.2:39485)
0926 11:44:16.259989 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:16.259997 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:16.260015 (+ 18us) server_negotiation.cc:407] Connection header received
0926 11:44:16.260090 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:16.260094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:16.260280 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:16.260664 (+ 384us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:16.260793 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.261779 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.263124 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.263860 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.263955 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:16.264092 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:16.264207 (+ 115us) server_negotiation.cc:299] Negotiation successful
0926 11:44:16.264299 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":108,"threads_started":1}
I20260926 11:44:16.264622 25079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:16.258662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48974 (local address 127.6.96.1:42347)
0926 11:44:16.259076 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:16.259092 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:16.259115 (+ 23us) server_negotiation.cc:407] Connection header received
0926 11:44:16.259397 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:16.259403 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:16.259639 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:16.259791 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:16.260285 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.261291 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.262634 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.263472 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.263576 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:16.263715 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:16.264398 (+ 683us) server_negotiation.cc:299] Negotiation successful
0926 11:44:16.264490 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":93,"threads_started":1}
I20260926 11:44:16.264436 25082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:16.259272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:39485 (local address 127.0.0.1:34176)
0926 11:44:16.259618 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:16.259630 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:16.259694 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:16.260423 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:16.260426 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:16.260441 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:16.260523 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:16.260528 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.261933 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.261937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:16.263002 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:16.263012 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.263973 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.263977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:16.264066 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:16.264097 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 11:44:16.264172 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 11:44:16.264246 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":91,"threads_started":1}
I20260926 11:44:16.264976 25081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:16.258799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:45043 (local address 127.0.0.1:41930)
0926 11:44:16.259477 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:16.259490 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:16.259586 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:16.260855 (+ 1269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:16.260858 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:16.260874 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:16.260958 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:16.260964 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.262196 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.262200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:16.263198 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:16.263208 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.264624 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.264626 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:16.264719 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:16.264743 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 11:44:16.264811 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 11:44:16.264871 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":90,"threads_started":1}
I20260926 11:44:16.265112 25080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:16.259053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41930 (local address 127.6.96.3:45043)
0926 11:44:16.260282 (+ 1229us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:16.260294 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:16.260315 (+ 21us) server_negotiation.cc:407] Connection header received
0926 11:44:16.260383 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:16.260388 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:16.260553 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:16.260698 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:16.261085 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.262081 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.263333 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.264093 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.264191 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:16.264315 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:16.264860 (+ 545us) server_negotiation.cc:299] Negotiation successful
0926 11:44:16.264959 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1117,"thread_start_us":106,"threads_started":1}
I20260926 11:44:16.267632 24733 tablet_service.cc:1467] Processing CreateTablet for tablet 0718173c4f4942b48edd58740dac8fb3 (DEFAULT_TABLE table=test-table [id=632536accd384e6ea557e48dc22671f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:44:16.267678 24867 tablet_service.cc:1467] Processing CreateTablet for tablet 0718173c4f4942b48edd58740dac8fb3 (DEFAULT_TABLE table=test-table [id=632536accd384e6ea557e48dc22671f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:44:16.267678 25001 tablet_service.cc:1467] Processing CreateTablet for tablet 0718173c4f4942b48edd58740dac8fb3 (DEFAULT_TABLE table=test-table [id=632536accd384e6ea557e48dc22671f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:44:16.268549 24733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0718173c4f4942b48edd58740dac8fb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:16.268666 24867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0718173c4f4942b48edd58740dac8fb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:16.268666 25001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0718173c4f4942b48edd58740dac8fb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:16.276127 25084 tablet_bootstrap.cc:492] T 0718173c4f4942b48edd58740dac8fb3 P 38d076b82e8949edbe0c813912533768: Bootstrap starting.
I20260926 11:44:16.276304 25085 tablet_bootstrap.cc:492] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d: Bootstrap starting.
I20260926 11:44:16.278015 25085 tablet_bootstrap.cc:654] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:16.278023 25084 tablet_bootstrap.cc:654] T 0718173c4f4942b48edd58740dac8fb3 P 38d076b82e8949edbe0c813912533768: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:16.279040 25084 log.cc:824] T 0718173c4f4942b48edd58740dac8fb3 P 38d076b82e8949edbe0c813912533768: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:16.279047 25085 log.cc:824] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:16.279214 25086 tablet_bootstrap.cc:492] T 0718173c4f4942b48edd58740dac8fb3 P 3de99b0ab95b426c892a9e24c3e4f6dc: Bootstrap starting.
I20260926 11:44:16.281309 25084 tablet_bootstrap.cc:492] T 0718173c4f4942b48edd58740dac8fb3 P 38d076b82e8949edbe0c813912533768: No bootstrap required, opened a new log
I20260926 11:44:16.281342 25086 tablet_bootstrap.cc:654] T 0718173c4f4942b48edd58740dac8fb3 P 3de99b0ab95b426c892a9e24c3e4f6dc: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:16.281647 25084 ts_tablet_manager.cc:1397] T 0718173c4f4942b48edd58740dac8fb3 P 38d076b82e8949edbe0c813912533768: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260926 11:44:16.282160 25085 tablet_bootstrap.cc:492] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d: No bootstrap required, opened a new log
I20260926 11:44:16.282343 25085 ts_tablet_manager.cc:1397] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260926 11:44:16.282580 25086 log.cc:824] T 0718173c4f4942b48edd58740dac8fb3 P 3de99b0ab95b426c892a9e24c3e4f6dc: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:16.284853 25086 tablet_bootstrap.cc:492] T 0718173c4f4942b48edd58740dac8fb3 P 3de99b0ab95b426c892a9e24c3e4f6dc: No bootstrap required, opened a new log
I20260926 11:44:16.285040 25086 ts_tablet_manager.cc:1397] T 0718173c4f4942b48edd58740dac8fb3 P 3de99b0ab95b426c892a9e24c3e4f6dc: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 11:44:16.285591 25085 raft_consensus.cc:348] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d076b82e8949edbe0c813912533768" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42347 } } peers { permanent_uuid: "488691a83e00473daa5a8f2adb837c4d" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 39485 } } peers { permanent_uuid: "3de99b0ab95b426c892a9e24c3e4f6dc" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 45043 } }
I20260926 11:44:16.285607 25084 raft_consensus.cc:348] T 0718173c4f4942b48edd58740dac8fb3 P 38d076b82e8949edbe0c813912533768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d076b82e8949edbe0c813912533768" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42347 } } peers { permanent_uuid: "488691a83e00473daa5a8f2adb837c4d" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 39485 } } peers { permanent_uuid: "3de99b0ab95b426c892a9e24c3e4f6dc" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 45043 } }
I20260926 11:44:16.285830 25085 raft_consensus.cc:374] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:16.285859 25084 raft_consensus.cc:374] T 0718173c4f4942b48edd58740dac8fb3 P 38d076b82e8949edbe0c813912533768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:16.285912 25085 raft_consensus.cc:729] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 488691a83e00473daa5a8f2adb837c4d, State: Initialized, Role: FOLLOWER
I20260926 11:44:16.285956 25084 raft_consensus.cc:729] T 0718173c4f4942b48edd58740dac8fb3 P 38d076b82e8949edbe0c813912533768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38d076b82e8949edbe0c813912533768, State: Initialized, Role: FOLLOWER
I20260926 11:44:16.286348 25085 consensus_queue.cc:260] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d076b82e8949edbe0c813912533768" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42347 } } peers { permanent_uuid: "488691a83e00473daa5a8f2adb837c4d" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 39485 } } peers { permanent_uuid: "3de99b0ab95b426c892a9e24c3e4f6dc" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 45043 } }
I20260926 11:44:16.286438 25084 consensus_queue.cc:260] T 0718173c4f4942b48edd58740dac8fb3 P 38d076b82e8949edbe0c813912533768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d076b82e8949edbe0c813912533768" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42347 } } peers { permanent_uuid: "488691a83e00473daa5a8f2adb837c4d" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 39485 } } peers { permanent_uuid: "3de99b0ab95b426c892a9e24c3e4f6dc" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 45043 } }
I20260926 11:44:16.286989 24933 heartbeater.cc:503] Master 127.6.96.62:34925 was elected leader, sending a full tablet report...
I20260926 11:44:16.287494 25084 ts_tablet_manager.cc:1428] T 0718173c4f4942b48edd58740dac8fb3 P 38d076b82e8949edbe0c813912533768: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20260926 11:44:16.287881 24799 heartbeater.cc:503] Master 127.6.96.62:34925 was elected leader, sending a full tablet report...
W20260926 11:44:16.288338 24800 tablet.cc:2369] T 0718173c4f4942b48edd58740dac8fb3 P 38d076b82e8949edbe0c813912533768: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:44:16.288427 25085 ts_tablet_manager.cc:1428] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260926 11:44:16.289654 25086 raft_consensus.cc:348] T 0718173c4f4942b48edd58740dac8fb3 P 3de99b0ab95b426c892a9e24c3e4f6dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d076b82e8949edbe0c813912533768" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42347 } } peers { permanent_uuid: "488691a83e00473daa5a8f2adb837c4d" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 39485 } } peers { permanent_uuid: "3de99b0ab95b426c892a9e24c3e4f6dc" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 45043 } }
I20260926 11:44:16.289870 25086 raft_consensus.cc:374] T 0718173c4f4942b48edd58740dac8fb3 P 3de99b0ab95b426c892a9e24c3e4f6dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:16.289937 25086 raft_consensus.cc:729] T 0718173c4f4942b48edd58740dac8fb3 P 3de99b0ab95b426c892a9e24c3e4f6dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3de99b0ab95b426c892a9e24c3e4f6dc, State: Initialized, Role: FOLLOWER
I20260926 11:44:16.290421 25086 consensus_queue.cc:260] T 0718173c4f4942b48edd58740dac8fb3 P 3de99b0ab95b426c892a9e24c3e4f6dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d076b82e8949edbe0c813912533768" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42347 } } peers { permanent_uuid: "488691a83e00473daa5a8f2adb837c4d" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 39485 } } peers { permanent_uuid: "3de99b0ab95b426c892a9e24c3e4f6dc" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 45043 } }
I20260926 11:44:16.291054 25067 heartbeater.cc:503] Master 127.6.96.62:34925 was elected leader, sending a full tablet report...
I20260926 11:44:16.291399 25086 ts_tablet_manager.cc:1428] T 0718173c4f4942b48edd58740dac8fb3 P 3de99b0ab95b426c892a9e24c3e4f6dc: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 11:44:16.294517 25090 raft_consensus.cc:482] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:44:16.294657 25090 raft_consensus.cc:504] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d076b82e8949edbe0c813912533768" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42347 } } peers { permanent_uuid: "488691a83e00473daa5a8f2adb837c4d" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 39485 } } peers { permanent_uuid: "3de99b0ab95b426c892a9e24c3e4f6dc" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 45043 } }
I20260926 11:44:16.295645 25090 leader_election.cc:288] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38d076b82e8949edbe0c813912533768 (127.6.96.1:42347), 3de99b0ab95b426c892a9e24c3e4f6dc (127.6.96.3:45043)
I20260926 11:44:16.299485 25080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:16.295998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:57991 (local address 127.6.96.3:45043)
0926 11:44:16.296426 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:16.296429 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:16.296440 (+ 11us) server_negotiation.cc:407] Connection header received
0926 11:44:16.296473 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:16.296475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:16.296506 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:16.296564 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:16.296817 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.297343 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.298560 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.298985 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.299074 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:16.299133 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:16.299305 (+ 172us) server_negotiation.cc:299] Negotiation successful
0926 11:44:16.299377 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345}
I20260926 11:44:16.299484 25079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:16.295884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:52269 (local address 127.6.96.1:42347)
0926 11:44:16.296116 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:16.296120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:16.296137 (+ 17us) server_negotiation.cc:407] Connection header received
0926 11:44:16.296179 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:16.296181 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:16.296217 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:16.296277 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:16.296525 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.297023 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.298500 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.298946 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.299045 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:16.299108 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:16.299295 (+ 187us) server_negotiation.cc:299] Negotiation successful
0926 11:44:16.299363 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157}
I20260926 11:44:16.299484 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:16.295831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:42347 (local address 127.6.96.2:52269)
0926 11:44:16.295986 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:16.296000 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:16.296095 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:16.296303 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:16.296305 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:16.296319 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:16.296423 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:16.296428 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.297146 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.297149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:16.298373 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:16.298382 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.299056 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.299059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:16.299177 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:16.299202 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 11:44:16.299268 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 11:44:16.299341 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"mutex_wait_us":24}
I20260926 11:44:16.299500 25093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:16.295858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:45043 (local address 127.6.96.2:57991)
0926 11:44:16.296187 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:16.296198 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:16.296262 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:16.296595 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:16.296598 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:16.296613 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:16.296721 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:16.296726 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.297449 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.297452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:16.298454 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:16.298464 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.299086 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.299090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:16.299194 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:16.299216 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 11:44:16.299281 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 11:44:16.299355 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":25,"thread_start_us":116,"threads_started":1}
I20260926 11:44:16.300067 25021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0718173c4f4942b48edd58740dac8fb3" candidate_uuid: "488691a83e00473daa5a8f2adb837c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3de99b0ab95b426c892a9e24c3e4f6dc" is_pre_election: true
I20260926 11:44:16.300074 24753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0718173c4f4942b48edd58740dac8fb3" candidate_uuid: "488691a83e00473daa5a8f2adb837c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38d076b82e8949edbe0c813912533768" is_pre_election: true
I20260926 11:44:16.300343 25021 raft_consensus.cc:2454] T 0718173c4f4942b48edd58740dac8fb3 P 3de99b0ab95b426c892a9e24c3e4f6dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 488691a83e00473daa5a8f2adb837c4d in term 0.
I20260926 11:44:16.300343 24753 raft_consensus.cc:2454] T 0718173c4f4942b48edd58740dac8fb3 P 38d076b82e8949edbe0c813912533768 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 488691a83e00473daa5a8f2adb837c4d in term 0.
I20260926 11:44:16.300748 24823 leader_election.cc:302] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3de99b0ab95b426c892a9e24c3e4f6dc, 488691a83e00473daa5a8f2adb837c4d; no voters:
I20260926 11:44:16.300993 25090 raft_consensus.cc:2781] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:44:16.301071 25090 raft_consensus.cc:482] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:44:16.301103 25090 raft_consensus.cc:3037] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:16.302112 25090 raft_consensus.cc:504] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d076b82e8949edbe0c813912533768" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42347 } } peers { permanent_uuid: "488691a83e00473daa5a8f2adb837c4d" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 39485 } } peers { permanent_uuid: "3de99b0ab95b426c892a9e24c3e4f6dc" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 45043 } }
I20260926 11:44:16.302448 25090 leader_election.cc:288] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d [CANDIDATE]: Term 1 election: Requested vote from peers 38d076b82e8949edbe0c813912533768 (127.6.96.1:42347), 3de99b0ab95b426c892a9e24c3e4f6dc (127.6.96.3:45043)
I20260926 11:44:16.302802 25021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0718173c4f4942b48edd58740dac8fb3" candidate_uuid: "488691a83e00473daa5a8f2adb837c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3de99b0ab95b426c892a9e24c3e4f6dc"
I20260926 11:44:16.302802 24753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0718173c4f4942b48edd58740dac8fb3" candidate_uuid: "488691a83e00473daa5a8f2adb837c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38d076b82e8949edbe0c813912533768"
I20260926 11:44:16.302947 25021 raft_consensus.cc:3037] T 0718173c4f4942b48edd58740dac8fb3 P 3de99b0ab95b426c892a9e24c3e4f6dc [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:16.302995 24753 raft_consensus.cc:3037] T 0718173c4f4942b48edd58740dac8fb3 P 38d076b82e8949edbe0c813912533768 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:16.304167 25021 raft_consensus.cc:2454] T 0718173c4f4942b48edd58740dac8fb3 P 3de99b0ab95b426c892a9e24c3e4f6dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 488691a83e00473daa5a8f2adb837c4d in term 1.
I20260926 11:44:16.304165 24753 raft_consensus.cc:2454] T 0718173c4f4942b48edd58740dac8fb3 P 38d076b82e8949edbe0c813912533768 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 488691a83e00473daa5a8f2adb837c4d in term 1.
I20260926 11:44:16.304466 24821 leader_election.cc:302] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38d076b82e8949edbe0c813912533768, 488691a83e00473daa5a8f2adb837c4d; no voters:
I20260926 11:44:16.304615 25090 raft_consensus.cc:2781] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:16.304826 25090 raft_consensus.cc:686] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d [term 1 LEADER]: Becoming Leader. State: Replica: 488691a83e00473daa5a8f2adb837c4d, State: Running, Role: LEADER
I20260926 11:44:16.305133 25090 consensus_queue.cc:237] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d076b82e8949edbe0c813912533768" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42347 } } peers { permanent_uuid: "488691a83e00473daa5a8f2adb837c4d" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 39485 } } peers { permanent_uuid: "3de99b0ab95b426c892a9e24c3e4f6dc" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 45043 } }
I20260926 11:44:16.308327 24607 catalog_manager.cc:5495] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d reported cstate change: term changed from 0 to 1, leader changed from <none> to 488691a83e00473daa5a8f2adb837c4d (127.6.96.2). New cstate: current_term: 1 leader_uuid: "488691a83e00473daa5a8f2adb837c4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d076b82e8949edbe0c813912533768" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "488691a83e00473daa5a8f2adb837c4d" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 39485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3de99b0ab95b426c892a9e24c3e4f6dc" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 45043 } health_report { overall_health: UNKNOWN } } }
I20260926 11:44:16.328727 25083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:16.325638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34178 (local address 127.6.96.2:39485)
0926 11:44:16.325748 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:16.325751 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:16.325764 (+ 13us) server_negotiation.cc:407] Connection header received
0926 11:44:16.325836 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:16.325838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:16.325891 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:16.325974 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:44:16.326142 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.326627 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.327454 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.327670 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.327757 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:16.328002 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:44:16.328406 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:44:16.328514 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:16.328568 (+ 54us) server_negotiation.cc:299] Negotiation successful
0926 11:44:16.328600 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 11:44:16.334484 25021 raft_consensus.cc:1261] T 0718173c4f4942b48edd58740dac8fb3 P 3de99b0ab95b426c892a9e24c3e4f6dc [term 1 FOLLOWER]: Refusing update from remote peer 488691a83e00473daa5a8f2adb837c4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:44:16.334509 24753 raft_consensus.cc:1261] T 0718173c4f4942b48edd58740dac8fb3 P 38d076b82e8949edbe0c813912533768 [term 1 FOLLOWER]: Refusing update from remote peer 488691a83e00473daa5a8f2adb837c4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:44:16.335119 25090 consensus_queue.cc:1035] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "38d076b82e8949edbe0c813912533768" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 42347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:44:16.335300 25094 consensus_queue.cc:1035] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3de99b0ab95b426c892a9e24c3e4f6dc" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 45043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:44:16.345150 25098 mvcc.cc:205] Tried to move back new op lower bound from 7333572838738960384 to 7333572838628536320. Current Snapshot: MvccSnapshot[applied={T|T < 7333572838738960384}]
I20260926 11:44:16.350689 24607 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:44:16.350894 24607 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:44:16.353142 24607 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 16f9135b6ce6467bb9422f81b4503543: Sending DeleteTablet for 3 replicas of tablet 0718173c4f4942b48edd58740dac8fb3
I20260926 11:44:16.353818 25001 tablet_service.cc:1514] Processing DeleteTablet for tablet 0718173c4f4942b48edd58740dac8fb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:44:16 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:41930
I20260926 11:44:16.354001 24733 tablet_service.cc:1514] Processing DeleteTablet for tablet 0718173c4f4942b48edd58740dac8fb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:44:16 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:48974
I20260926 11:44:16.354331 25106 tablet_replica.cc:330] T 0718173c4f4942b48edd58740dac8fb3 P 3de99b0ab95b426c892a9e24c3e4f6dc: stopping tablet replica
I20260926 11:44:16.354418 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 24670
I20260926 11:44:16.354444 24867 tablet_service.cc:1514] Processing DeleteTablet for tablet 0718173c4f4942b48edd58740dac8fb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:44:16 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:34176
I20260926 11:44:16.354652 25106 raft_consensus.cc:2229] T 0718173c4f4942b48edd58740dac8fb3 P 3de99b0ab95b426c892a9e24c3e4f6dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:44:16.354959 25106 raft_consensus.cc:2258] T 0718173c4f4942b48edd58740dac8fb3 P 3de99b0ab95b426c892a9e24c3e4f6dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:44:16.356334 25106 ts_tablet_manager.cc:1905] T 0718173c4f4942b48edd58740dac8fb3 P 3de99b0ab95b426c892a9e24c3e4f6dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:44:16.358675 25106 ts_tablet_manager.cc:1918] T 0718173c4f4942b48edd58740dac8fb3 P 3de99b0ab95b426c892a9e24c3e4f6dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:44:16.358762 25106 log.cc:1196] T 0718173c4f4942b48edd58740dac8fb3 P 3de99b0ab95b426c892a9e24c3e4f6dc: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/0718173c4f4942b48edd58740dac8fb3
I20260926 11:44:16.359059 25106 ts_tablet_manager.cc:1939] T 0718173c4f4942b48edd58740dac8fb3 P 3de99b0ab95b426c892a9e24c3e4f6dc: Deleting consensus metadata
I20260926 11:44:16.360064 24595 catalog_manager.cc:4841] TS 3de99b0ab95b426c892a9e24c3e4f6dc (127.6.96.3:45043): tablet 0718173c4f4942b48edd58740dac8fb3 (table test-table [id=632536accd384e6ea557e48dc22671f2]) successfully deleted
W20260926 11:44:16.362440 24593 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.96.1:42347 (error 108)
W20260926 11:44:16.364118 25078 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:16.363169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:42347 (local address 127.0.0.1:48986)
0926 11:44:16.363346 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:16.363360 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:16.363436 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:16.364014 (+ 578us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.1:42347: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":99}
I20260926 11:44:16.364274 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 24804
I20260926 11:44:16.364320 25108 tablet_replica.cc:330] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d: stopping tablet replica
W20260926 11:44:16.364403 24593 catalog_manager.cc:4584] TS 38d076b82e8949edbe0c813912533768 (127.6.96.1:42347): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0718173c4f4942b48edd58740dac8fb3: Network error: Client connection negotiation failed: client connection to 127.6.96.1:42347: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 11:44:16.364619 25108 raft_consensus.cc:2229] T 0718173c4f4942b48edd58740dac8fb3 P 488691a83e00473daa5a8f2adb837c4d [term 1 LEADER]: Raft consensus shutting down.
I20260926 11:44:16.369194 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 24938
I20260926 11:44:16.370144 25078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:16.369355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:39485 (local address 127.0.0.1:34188)
0926 11:44:16.369991 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:16.370068 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.2:39485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":566}
I20260926 11:44:16.373606 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 24576
2026-09-26T11:44:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2111 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-26T11:44:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:16Z Disabled control of system clock
I20260926 11:44:16.401453 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:36835
--webserver_interface=127.6.96.62
--webserver_port=0
--builtin_ntp_servers=127.6.96.20:44683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:36835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 11:44:16.508492 25113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:16.508832 25113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:16.508872 25113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:16.513123 25113 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:16.513195 25113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:16.513208 25113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:16.513224 25113 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:16.513238 25113 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:16.518567 25113 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:44683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:36835
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:36835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:16.519670 25113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:16.521013 25113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:16.529075 25118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:16.529140 25119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:16.529189 25113 server_base.cc:1034] running on GCE node
W20260926 11:44:16.529146 25121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:16.529850 25113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:16.531028 25113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:16.532186 25113 hybrid_clock.cc:648] HybridClock initialized: now 1790423056532172 us; error 25 us; skew 500 ppm
I20260926 11:44:16.533969 25113 webserver.cc:460] Webserver started at http://127.6.96.62:43975/ using document root <none> and password file <none>
I20260926 11:44:16.534502 25113 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:16.534560 25113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:16.534722 25113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:16.536736 25113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "999c6a2262ee49f7a54786a8750c87c2"
format_stamp: "Formatted at 2026-09-26 11:44:16 on dist-test-slave-xxxx"
server_key: "8c04b45a963641d63a903b2374a5f90d"
server_key_iv: "4d0ba651f8124fdbe7f42917045a02e9"
server_key_version: "encryptionkey@0"
I20260926 11:44:16.537247 25113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "999c6a2262ee49f7a54786a8750c87c2"
format_stamp: "Formatted at 2026-09-26 11:44:16 on dist-test-slave-xxxx"
server_key: "8c04b45a963641d63a903b2374a5f90d"
server_key_iv: "4d0ba651f8124fdbe7f42917045a02e9"
server_key_version: "encryptionkey@0"
I20260926 11:44:16.540861 25113 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 11:44:16.543267 25127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:16.544500 25113 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:44:16.544605 25113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "999c6a2262ee49f7a54786a8750c87c2"
format_stamp: "Formatted at 2026-09-26 11:44:16 on dist-test-slave-xxxx"
server_key: "8c04b45a963641d63a903b2374a5f90d"
server_key_iv: "4d0ba651f8124fdbe7f42917045a02e9"
server_key_version: "encryptionkey@0"
I20260926 11:44:16.544716 25113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:16.586105 25113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:16.586910 25113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:16.587106 25113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:16.593962 25113 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:36835
I20260926 11:44:16.593977 25179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:36835 every 8 connection(s)
I20260926 11:44:16.594975 25113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:16.597380 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 25113
I20260926 11:44:16.597524 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:16.597815 6528 external_mini_cluster.cc:1428] Setting key a62e9e70bc1c6bfc10ba11095e8fd327
I20260926 11:44:16.598384 25180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:16.604493 25180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 999c6a2262ee49f7a54786a8750c87c2: Bootstrap starting.
I20260926 11:44:16.606798 25183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:16.599223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42838 (local address 127.6.96.62:36835)
0926 11:44:16.599873 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:16.599887 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:16.599923 (+ 36us) server_negotiation.cc:407] Connection header received
0926 11:44:16.600597 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:16.600614 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:16.601004 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:16.601321 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:16.601737 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.602695 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.603592 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.604060 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.604209 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:16.605066 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:16.605077 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:16.605090 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:16.605462 (+ 372us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:16.605516 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:16.605536 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:16.605620 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:16.605765 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 11:44:16.606022 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":147,"threads_started":1}
I20260926 11:44:16.606896 25180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 999c6a2262ee49f7a54786a8750c87c2: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:16.607649 25180 log.cc:824] T 00000000000000000000000000000000 P 999c6a2262ee49f7a54786a8750c87c2: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:16.609552 25180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 999c6a2262ee49f7a54786a8750c87c2: No bootstrap required, opened a new log
I20260926 11:44:16.613161 25180 raft_consensus.cc:348] T 00000000000000000000000000000000 P 999c6a2262ee49f7a54786a8750c87c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "999c6a2262ee49f7a54786a8750c87c2" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36835 } }
I20260926 11:44:16.613402 25180 raft_consensus.cc:374] T 00000000000000000000000000000000 P 999c6a2262ee49f7a54786a8750c87c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:16.613461 25180 raft_consensus.cc:729] T 00000000000000000000000000000000 P 999c6a2262ee49f7a54786a8750c87c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 999c6a2262ee49f7a54786a8750c87c2, State: Initialized, Role: FOLLOWER
I20260926 11:44:16.613881 25180 consensus_queue.cc:260] T 00000000000000000000000000000000 P 999c6a2262ee49f7a54786a8750c87c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "999c6a2262ee49f7a54786a8750c87c2" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36835 } }
I20260926 11:44:16.614019 25180 raft_consensus.cc:388] T 00000000000000000000000000000000 P 999c6a2262ee49f7a54786a8750c87c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:44:16.614084 25180 raft_consensus.cc:482] T 00000000000000000000000000000000 P 999c6a2262ee49f7a54786a8750c87c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:44:16.614181 25180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 999c6a2262ee49f7a54786a8750c87c2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:16.615132 25180 raft_consensus.cc:504] T 00000000000000000000000000000000 P 999c6a2262ee49f7a54786a8750c87c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "999c6a2262ee49f7a54786a8750c87c2" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36835 } }
I20260926 11:44:16.615469 25180 leader_election.cc:302] T 00000000000000000000000000000000 P 999c6a2262ee49f7a54786a8750c87c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 999c6a2262ee49f7a54786a8750c87c2; no voters:
I20260926 11:44:16.615768 25180 leader_election.cc:288] T 00000000000000000000000000000000 P 999c6a2262ee49f7a54786a8750c87c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:44:16.615869 25185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 999c6a2262ee49f7a54786a8750c87c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:16.616098 25185 raft_consensus.cc:686] T 00000000000000000000000000000000 P 999c6a2262ee49f7a54786a8750c87c2 [term 1 LEADER]: Becoming Leader. State: Replica: 999c6a2262ee49f7a54786a8750c87c2, State: Running, Role: LEADER
I20260926 11:44:16.616387 25185 consensus_queue.cc:237] T 00000000000000000000000000000000 P 999c6a2262ee49f7a54786a8750c87c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "999c6a2262ee49f7a54786a8750c87c2" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36835 } }
I20260926 11:44:16.616935 25180 sys_catalog.cc:564] T 00000000000000000000000000000000 P 999c6a2262ee49f7a54786a8750c87c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:16.618067 25187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 999c6a2262ee49f7a54786a8750c87c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 999c6a2262ee49f7a54786a8750c87c2. Latest consensus state: current_term: 1 leader_uuid: "999c6a2262ee49f7a54786a8750c87c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "999c6a2262ee49f7a54786a8750c87c2" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36835 } } }
I20260926 11:44:16.618197 25187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 999c6a2262ee49f7a54786a8750c87c2 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:16.618484 25186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 999c6a2262ee49f7a54786a8750c87c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "999c6a2262ee49f7a54786a8750c87c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "999c6a2262ee49f7a54786a8750c87c2" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36835 } } }
I20260926 11:44:16.618561 25194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:16.619092 25186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 999c6a2262ee49f7a54786a8750c87c2 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:16.621752 25194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:16.627503 25194 catalog_manager.cc:1347] Generated new cluster ID: bb1997e9c957419496051b57e97ba8e3
I20260926 11:44:16.627576 25194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:16.652148 25194 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:16.653483 25194 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:16.660022 25194 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 999c6a2262ee49f7a54786a8750c87c2: Generated new TSK 0
I20260926 11:44:16.660836 25194 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:44:16.665758 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:36835
--builtin_ntp_servers=127.6.96.20:44683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 11:44:16.777313 25204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:16.777556 25204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:16.777592 25204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:16.781742 25204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:16.781807 25204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:16.781860 25204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:44:16.786558 25204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:44683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:36835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:16.787727 25204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:16.789163 25204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:16.798818 25212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:16.798801 25210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:16.798909 25209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:16.799110 25204 server_base.cc:1034] running on GCE node
I20260926 11:44:16.799618 25204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:16.800320 25204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:16.801499 25204 hybrid_clock.cc:648] HybridClock initialized: now 1790423056801479 us; error 40 us; skew 500 ppm
I20260926 11:44:16.803472 25204 webserver.cc:460] Webserver started at http://127.6.96.1:45445/ using document root <none> and password file <none>
I20260926 11:44:16.804033 25204 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:16.804103 25204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:16.804292 25204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:16.806373 25204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "14d13e37020a4a5e851854ea56a6f743"
format_stamp: "Formatted at 2026-09-26 11:44:16 on dist-test-slave-xxxx"
server_key: "afd281d100c438f63a49949706fae0fe"
server_key_iv: "47a4f08a8c6942e476e4409ae1acb342"
server_key_version: "encryptionkey@0"
I20260926 11:44:16.806977 25204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "14d13e37020a4a5e851854ea56a6f743"
format_stamp: "Formatted at 2026-09-26 11:44:16 on dist-test-slave-xxxx"
server_key: "afd281d100c438f63a49949706fae0fe"
server_key_iv: "47a4f08a8c6942e476e4409ae1acb342"
server_key_version: "encryptionkey@0"
I20260926 11:44:16.810742 25204 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 11:44:16.813289 25218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:16.814548 25204 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:44:16.814661 25204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "14d13e37020a4a5e851854ea56a6f743"
format_stamp: "Formatted at 2026-09-26 11:44:16 on dist-test-slave-xxxx"
server_key: "afd281d100c438f63a49949706fae0fe"
server_key_iv: "47a4f08a8c6942e476e4409ae1acb342"
server_key_version: "encryptionkey@0"
I20260926 11:44:16.814769 25204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:16.825364 25204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:16.826036 25204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:16.826217 25204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:16.826778 25204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:44:16.827755 25204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:44:16.827804 25204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:16.827857 25204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:44:16.827890 25204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:16.836176 25204 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:38499
I20260926 11:44:16.836205 25331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:38499 every 8 connection(s)
I20260926 11:44:16.837045 25204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:44:16.841938 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 25204
I20260926 11:44:16.842061 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:44:16.842326 6528 external_mini_cluster.cc:1428] Setting key 85f8abfb2aee12dc1063bebd2cd0cad4
I20260926 11:44:16.844393 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:36835
--builtin_ntp_servers=127.6.96.20:44683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 11:44:16.846719 25183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:16.838780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:56703 (local address 127.6.96.62:36835)
0926 11:44:16.838942 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:16.838946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:16.840013 (+ 1067us) server_negotiation.cc:407] Connection header received
0926 11:44:16.841360 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:16.841364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:16.841446 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:16.841519 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:16.842485 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.843202 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.844677 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.844993 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.845112 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:16.846239 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:16.846243 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:16.846245 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:16.846260 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:16.846284 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:16.846294 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:16.846370 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:16.846500 (+ 130us) server_negotiation.cc:299] Negotiation successful
0926 11:44:16.846575 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 11:44:16.847553 25334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:16.839207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:36835 (local address 127.6.96.1:56703)
0926 11:44:16.839835 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:16.839908 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:16.841149 (+ 1241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:16.841701 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:16.841708 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:16.841795 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:16.842332 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:16.842347 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.843347 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.843351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:16.844501 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:16.844514 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:16.845191 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:16.845195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:16.845353 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:16.845966 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 11:44:16.845968 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:16.846017 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 11:44:16.846087 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:16.846373 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:16.846379 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:16.846382 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 11:44:16.846484 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 11:44:16.846720 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":103,"threads_started":1}
I20260926 11:44:16.848855 25332 heartbeater.cc:347] Connected to a master server at 127.6.96.62:36835
I20260926 11:44:16.849126 25332 heartbeater.cc:464] Registering TS with master...
I20260926 11:44:16.849664 25332 heartbeater.cc:511] Master 127.6.96.62:36835 requested a full tablet report, sending...
I20260926 11:44:16.851390 25143 ts_manager.cc:194] Registered new tserver with Master: 14d13e37020a4a5e851854ea56a6f743 (127.6.96.1:38499)
I20260926 11:44:16.852989 25143 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.96.1:56703
W20260926 11:44:16.955856 25335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:16.956177 25335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:16.956235 25335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:16.960875 25335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:16.960934 25335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:16.960992 25335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:44:16.965770 25335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:44683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:36835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:16.966922 25335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:16.968286 25335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:16.978611 25341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:16.978641 25340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:16.978739 25335 server_base.cc:1034] running on GCE node
W20260926 11:44:16.978662 25343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:16.979341 25335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:16.980046 25335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:16.981247 25335 hybrid_clock.cc:648] HybridClock initialized: now 1790423056981215 us; error 47 us; skew 500 ppm
I20260926 11:44:16.983265 25335 webserver.cc:460] Webserver started at http://127.6.96.2:40443/ using document root <none> and password file <none>
I20260926 11:44:16.983815 25335 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:16.983876 25335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:16.984055 25335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:16.986239 25335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "b1d51012b6e348d383a80b5b8bc84312"
format_stamp: "Formatted at 2026-09-26 11:44:16 on dist-test-slave-xxxx"
server_key: "c8ba377dcc2a6654108882c5bbe56cf8"
server_key_iv: "2007cbd0de454f502e4c9cd3caf2662e"
server_key_version: "encryptionkey@0"
I20260926 11:44:16.986779 25335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "b1d51012b6e348d383a80b5b8bc84312"
format_stamp: "Formatted at 2026-09-26 11:44:16 on dist-test-slave-xxxx"
server_key: "c8ba377dcc2a6654108882c5bbe56cf8"
server_key_iv: "2007cbd0de454f502e4c9cd3caf2662e"
server_key_version: "encryptionkey@0"
I20260926 11:44:16.990729 25335 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260926 11:44:16.993631 25349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:16.995249 25335 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:44:16.995384 25335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "b1d51012b6e348d383a80b5b8bc84312"
format_stamp: "Formatted at 2026-09-26 11:44:16 on dist-test-slave-xxxx"
server_key: "c8ba377dcc2a6654108882c5bbe56cf8"
server_key_iv: "2007cbd0de454f502e4c9cd3caf2662e"
server_key_version: "encryptionkey@0"
I20260926 11:44:16.995498 25335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:17.020439 25335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:17.021166 25335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:17.021322 25335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:17.021863 25335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:44:17.022764 25335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:44:17.022815 25335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:17.022889 25335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:44:17.022919 25335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:17.030985 25335 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:43879
I20260926 11:44:17.031006 25462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:43879 every 8 connection(s)
I20260926 11:44:17.031855 25335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:44:17.040731 25183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:17.033660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:53681 (local address 127.6.96.62:36835)
0926 11:44:17.033795 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:17.033798 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:17.034808 (+ 1010us) server_negotiation.cc:407] Connection header received
0926 11:44:17.036215 (+ 1407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:17.036218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:17.036276 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:17.036347 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:17.037254 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.037773 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.038930 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.039153 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.039241 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:17.040281 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:17.040284 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:17.040286 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:17.040298 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:17.040316 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:17.040324 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:17.040380 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:17.040515 (+ 135us) server_negotiation.cc:299] Negotiation successful
0926 11:44:17.040584 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 11:44:17.041182 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 25335
I20260926 11:44:17.041296 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:44:17.041579 6528 external_mini_cluster.cc:1428] Setting key e2901d57e6004c7e3aa2a8ef91cf46d2
I20260926 11:44:17.041626 25465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:17.034006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:36835 (local address 127.6.96.2:53681)
0926 11:44:17.034650 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:17.034691 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:17.035973 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:17.036525 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:17.036532 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:17.036619 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:17.037111 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:17.037124 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.037899 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.037902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:17.038789 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:17.038800 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.039293 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.039297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:17.039455 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:17.039999 (+ 544us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 11:44:17.040000 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:17.040046 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 11:44:17.040138 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:17.040389 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:17.040398 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:17.040404 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 11:44:17.040507 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 11:44:17.040780 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":108,"threads_started":1}
I20260926 11:44:17.042737 25463 heartbeater.cc:347] Connected to a master server at 127.6.96.62:36835
I20260926 11:44:17.042992 25463 heartbeater.cc:464] Registering TS with master...
I20260926 11:44:17.043493 25463 heartbeater.cc:511] Master 127.6.96.62:36835 requested a full tablet report, sending...
I20260926 11:44:17.043577 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:36835
--builtin_ntp_servers=127.6.96.20:44683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 11:44:17.044482 25144 ts_manager.cc:194] Registered new tserver with Master: b1d51012b6e348d383a80b5b8bc84312 (127.6.96.2:43879)
I20260926 11:44:17.045035 25144 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.96.2:53681
W20260926 11:44:17.153941 25466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:17.154214 25466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:17.154253 25466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:17.158315 25466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:17.158376 25466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:17.158430 25466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:44:17.163278 25466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:44683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:36835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:17.164366 25466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:17.165711 25466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 11:44:17.175210 25466 server_base.cc:1034] running on GCE node
W20260926 11:44:17.175168 25472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:17.175163 25474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:17.175227 25471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:17.175745 25466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:17.176502 25466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:17.177688 25466 hybrid_clock.cc:648] HybridClock initialized: now 1790423057177664 us; error 29 us; skew 500 ppm
I20260926 11:44:17.179575 25466 webserver.cc:460] Webserver started at http://127.6.96.3:37001/ using document root <none> and password file <none>
I20260926 11:44:17.180162 25466 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:17.180229 25466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:17.180418 25466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:17.182484 25466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "b893858014324896bd53816979101126"
format_stamp: "Formatted at 2026-09-26 11:44:17 on dist-test-slave-xxxx"
server_key: "3abcc49c3c864d469076c52c8edc63d2"
server_key_iv: "7b251f61eee114081313b7577f18bb55"
server_key_version: "encryptionkey@0"
I20260926 11:44:17.183133 25466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "b893858014324896bd53816979101126"
format_stamp: "Formatted at 2026-09-26 11:44:17 on dist-test-slave-xxxx"
server_key: "3abcc49c3c864d469076c52c8edc63d2"
server_key_iv: "7b251f61eee114081313b7577f18bb55"
server_key_version: "encryptionkey@0"
I20260926 11:44:17.187287 25466 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260926 11:44:17.190188 25480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:17.191605 25466 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:44:17.191728 25466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "b893858014324896bd53816979101126"
format_stamp: "Formatted at 2026-09-26 11:44:17 on dist-test-slave-xxxx"
server_key: "3abcc49c3c864d469076c52c8edc63d2"
server_key_iv: "7b251f61eee114081313b7577f18bb55"
server_key_version: "encryptionkey@0"
I20260926 11:44:17.191847 25466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:17.220584 25466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:17.221344 25466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:17.221529 25466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:17.222133 25466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:44:17.223143 25466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:44:17.223193 25466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:17.223249 25466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:44:17.223277 25466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:17.231678 25466 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:33133
I20260926 11:44:17.231750 25593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:33133 every 8 connection(s)
I20260926 11:44:17.232609 25466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:44:17.239817 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 25466
I20260926 11:44:17.239954 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:44:17.240272 6528 external_mini_cluster.cc:1428] Setting key 1096eeb616ac676cba5cef06a4f649f8
I20260926 11:44:17.242429 25183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:17.234459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:41973 (local address 127.6.96.62:36835)
0926 11:44:17.234608 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:17.234612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:17.235730 (+ 1118us) server_negotiation.cc:407] Connection header received
0926 11:44:17.237154 (+ 1424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:17.237157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:17.237218 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:17.237292 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:17.238291 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.238859 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.240293 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.240571 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.240693 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:17.241956 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:17.241960 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:17.241963 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:17.241976 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:17.242000 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:17.242009 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:17.242082 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:17.242213 (+ 131us) server_negotiation.cc:299] Negotiation successful
0926 11:44:17.242278 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 11:44:17.243407 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:17.234904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:36835 (local address 127.6.96.3:41973)
0926 11:44:17.235574 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:17.235618 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:17.236923 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:17.237483 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:17.237491 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:17.237598 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:17.238140 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:17.238155 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.238983 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.238986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:17.240139 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:17.240152 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.240761 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.240765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:17.240975 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:17.241636 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 11:44:17.241638 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:17.241698 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 11:44:17.241804 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:17.242086 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:17.242091 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:17.242096 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 11:44:17.242195 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 11:44:17.242482 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":102,"threads_started":1}
I20260926 11:44:17.244552 25594 heartbeater.cc:347] Connected to a master server at 127.6.96.62:36835
I20260926 11:44:17.244791 25594 heartbeater.cc:464] Registering TS with master...
I20260926 11:44:17.245352 25594 heartbeater.cc:511] Master 127.6.96.62:36835 requested a full tablet report, sending...
I20260926 11:44:17.246493 25144 ts_manager.cc:194] Registered new tserver with Master: b893858014324896bd53816979101126 (127.6.96.3:33133)
I20260926 11:44:17.247164 25144 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.96.3:41973
I20260926 11:44:17.254557 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 11:44:17.260370 25183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:17.256483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42846 (local address 127.6.96.62:36835)
0926 11:44:17.256628 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:17.256633 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:17.256772 (+ 139us) server_negotiation.cc:407] Connection header received
0926 11:44:17.256909 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:17.256912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:17.256956 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:17.257020 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:17.257312 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.257835 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.258676 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.258943 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.259033 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:17.259891 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:17.259895 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:17.259898 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:17.259910 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:17.259928 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:17.259935 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:17.260032 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:17.260189 (+ 157us) server_negotiation.cc:299] Negotiation successful
0926 11:44:17.260226 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 11:44:17.263147 25144 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42846:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 11:44:17.266013 25144 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:44:17.279644 25608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:17.274945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:43879 (local address 127.0.0.1:40680)
0926 11:44:17.275559 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:17.275572 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:17.275652 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:17.276116 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:17.276119 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:17.276137 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:17.276229 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:17.276235 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.277170 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.277173 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:17.278319 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:17.278329 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.279123 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.279127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:17.279224 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:17.279259 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 11:44:17.279369 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 11:44:17.279491 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":1,"thread_start_us":82,"threads_started":1}
I20260926 11:44:17.279647 25610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:17.275329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40680 (local address 127.6.96.2:43879)
0926 11:44:17.275719 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:17.275728 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:17.275748 (+ 20us) server_negotiation.cc:407] Connection header received
0926 11:44:17.275815 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:17.275819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:17.275989 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:17.276114 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:17.276338 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.277068 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.278430 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.279021 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.279114 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:17.279313 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:17.279398 (+ 85us) server_negotiation.cc:299] Negotiation successful
0926 11:44:17.279494 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":112,"threads_started":1}
I20260926 11:44:17.280546 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:17.274192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:38499 (local address 127.0.0.1:37928)
0926 11:44:17.274552 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:17.274597 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:17.274714 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:17.275954 (+ 1240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:17.275960 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:17.275984 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:17.276140 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:17.276149 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.277649 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.277652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:17.279047 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:17.279066 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.280148 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.280151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:17.280267 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:17.280288 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 11:44:17.280336 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 11:44:17.280406 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":119,"thread_start_us":89,"threads_started":1}
I20260926 11:44:17.280820 25606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:17.274400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:33133 (local address 127.0.0.1:39378)
0926 11:44:17.275142 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:17.275155 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:17.275239 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:17.276185 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:17.276188 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:17.276205 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:17.276279 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:17.276284 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.277649 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.277652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:17.279056 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:17.279066 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.280232 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.280235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:17.280373 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:17.280397 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 11:44:17.280470 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 11:44:17.280537 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"mutex_wait_us":181,"thread_start_us":86,"threads_started":1}
I20260926 11:44:17.281236 25607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:17.274343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37928 (local address 127.6.96.1:38499)
0926 11:44:17.275450 (+ 1107us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:17.275462 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:17.275487 (+ 25us) server_negotiation.cc:407] Connection header received
0926 11:44:17.275563 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:17.275569 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:17.275783 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:17.275951 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:17.276270 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.277518 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.279193 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.279994 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.280750 (+ 756us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:17.280934 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:17.280981 (+ 47us) server_negotiation.cc:299] Negotiation successful
0926 11:44:17.281085 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":996,"thread_start_us":141,"threads_started":1}
I20260926 11:44:17.283130 25609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:17.275021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39378 (local address 127.6.96.3:33133)
0926 11:44:17.275664 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:17.275675 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:17.275699 (+ 24us) server_negotiation.cc:407] Connection header received
0926 11:44:17.275769 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:17.275773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:17.275921 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:17.276056 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:17.276388 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.277531 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.279193 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.280090 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.280175 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:17.280359 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:17.280511 (+ 152us) server_negotiation.cc:299] Negotiation successful
0926 11:44:17.282963 (+ 2452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":145,"threads_started":1}
I20260926 11:44:17.282684 25397 tablet_service.cc:1467] Processing CreateTablet for tablet 057600b935cb42718995db2672241c78 (DEFAULT_TABLE table=test-table [id=348ad8a9485346f39b8d0dc90cab501c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:44:17.283793 25397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 057600b935cb42718995db2672241c78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:17.283919 25266 tablet_service.cc:1467] Processing CreateTablet for tablet 057600b935cb42718995db2672241c78 (DEFAULT_TABLE table=test-table [id=348ad8a9485346f39b8d0dc90cab501c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:44:17.284754 25266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 057600b935cb42718995db2672241c78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:17.285843 25528 tablet_service.cc:1467] Processing CreateTablet for tablet 057600b935cb42718995db2672241c78 (DEFAULT_TABLE table=test-table [id=348ad8a9485346f39b8d0dc90cab501c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:44:17.286914 25528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 057600b935cb42718995db2672241c78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:17.290989 25611 tablet_bootstrap.cc:492] T 057600b935cb42718995db2672241c78 P 14d13e37020a4a5e851854ea56a6f743: Bootstrap starting.
I20260926 11:44:17.293159 25611 tablet_bootstrap.cc:654] T 057600b935cb42718995db2672241c78 P 14d13e37020a4a5e851854ea56a6f743: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:17.293455 25613 tablet_bootstrap.cc:492] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126: Bootstrap starting.
I20260926 11:44:17.294005 25611 log.cc:824] T 057600b935cb42718995db2672241c78 P 14d13e37020a4a5e851854ea56a6f743: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:17.294027 25612 tablet_bootstrap.cc:492] T 057600b935cb42718995db2672241c78 P b1d51012b6e348d383a80b5b8bc84312: Bootstrap starting.
I20260926 11:44:17.295734 25613 tablet_bootstrap.cc:654] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:17.296073 25612 tablet_bootstrap.cc:654] T 057600b935cb42718995db2672241c78 P b1d51012b6e348d383a80b5b8bc84312: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:17.296542 25611 tablet_bootstrap.cc:492] T 057600b935cb42718995db2672241c78 P 14d13e37020a4a5e851854ea56a6f743: No bootstrap required, opened a new log
I20260926 11:44:17.296654 25613 log.cc:824] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:17.296737 25611 ts_tablet_manager.cc:1397] T 057600b935cb42718995db2672241c78 P 14d13e37020a4a5e851854ea56a6f743: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 11:44:17.296779 25612 log.cc:824] T 057600b935cb42718995db2672241c78 P b1d51012b6e348d383a80b5b8bc84312: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:17.298713 25612 tablet_bootstrap.cc:492] T 057600b935cb42718995db2672241c78 P b1d51012b6e348d383a80b5b8bc84312: No bootstrap required, opened a new log
I20260926 11:44:17.298733 25613 tablet_bootstrap.cc:492] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126: No bootstrap required, opened a new log
I20260926 11:44:17.298928 25612 ts_tablet_manager.cc:1397] T 057600b935cb42718995db2672241c78 P b1d51012b6e348d383a80b5b8bc84312: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260926 11:44:17.298928 25613 ts_tablet_manager.cc:1397] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 11:44:17.300056 25611 raft_consensus.cc:348] T 057600b935cb42718995db2672241c78 P 14d13e37020a4a5e851854ea56a6f743 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d13e37020a4a5e851854ea56a6f743" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38499 } } peers { permanent_uuid: "b1d51012b6e348d383a80b5b8bc84312" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 43879 } } peers { permanent_uuid: "b893858014324896bd53816979101126" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 33133 } }
I20260926 11:44:17.300292 25611 raft_consensus.cc:374] T 057600b935cb42718995db2672241c78 P 14d13e37020a4a5e851854ea56a6f743 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:17.300348 25611 raft_consensus.cc:729] T 057600b935cb42718995db2672241c78 P 14d13e37020a4a5e851854ea56a6f743 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14d13e37020a4a5e851854ea56a6f743, State: Initialized, Role: FOLLOWER
I20260926 11:44:17.300758 25611 consensus_queue.cc:260] T 057600b935cb42718995db2672241c78 P 14d13e37020a4a5e851854ea56a6f743 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d13e37020a4a5e851854ea56a6f743" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38499 } } peers { permanent_uuid: "b1d51012b6e348d383a80b5b8bc84312" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 43879 } } peers { permanent_uuid: "b893858014324896bd53816979101126" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 33133 } }
I20260926 11:44:17.301616 25611 ts_tablet_manager.cc:1428] T 057600b935cb42718995db2672241c78 P 14d13e37020a4a5e851854ea56a6f743: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260926 11:44:17.301960 25332 heartbeater.cc:503] Master 127.6.96.62:36835 was elected leader, sending a full tablet report...
I20260926 11:44:17.303133 25613 raft_consensus.cc:348] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d13e37020a4a5e851854ea56a6f743" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38499 } } peers { permanent_uuid: "b1d51012b6e348d383a80b5b8bc84312" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 43879 } } peers { permanent_uuid: "b893858014324896bd53816979101126" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 33133 } }
I20260926 11:44:17.303133 25612 raft_consensus.cc:348] T 057600b935cb42718995db2672241c78 P b1d51012b6e348d383a80b5b8bc84312 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d13e37020a4a5e851854ea56a6f743" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38499 } } peers { permanent_uuid: "b1d51012b6e348d383a80b5b8bc84312" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 43879 } } peers { permanent_uuid: "b893858014324896bd53816979101126" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 33133 } }
I20260926 11:44:17.303390 25612 raft_consensus.cc:374] T 057600b935cb42718995db2672241c78 P b1d51012b6e348d383a80b5b8bc84312 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:17.303390 25613 raft_consensus.cc:374] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:17.303435 25612 raft_consensus.cc:729] T 057600b935cb42718995db2672241c78 P b1d51012b6e348d383a80b5b8bc84312 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d51012b6e348d383a80b5b8bc84312, State: Initialized, Role: FOLLOWER
I20260926 11:44:17.303435 25613 raft_consensus.cc:729] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b893858014324896bd53816979101126, State: Initialized, Role: FOLLOWER
I20260926 11:44:17.303861 25612 consensus_queue.cc:260] T 057600b935cb42718995db2672241c78 P b1d51012b6e348d383a80b5b8bc84312 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d13e37020a4a5e851854ea56a6f743" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38499 } } peers { permanent_uuid: "b1d51012b6e348d383a80b5b8bc84312" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 43879 } } peers { permanent_uuid: "b893858014324896bd53816979101126" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 33133 } }
I20260926 11:44:17.304131 25613 consensus_queue.cc:260] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d13e37020a4a5e851854ea56a6f743" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38499 } } peers { permanent_uuid: "b1d51012b6e348d383a80b5b8bc84312" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 43879 } } peers { permanent_uuid: "b893858014324896bd53816979101126" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 33133 } }
I20260926 11:44:17.304566 25463 heartbeater.cc:503] Master 127.6.96.62:36835 was elected leader, sending a full tablet report...
I20260926 11:44:17.304836 25612 ts_tablet_manager.cc:1428] T 057600b935cb42718995db2672241c78 P b1d51012b6e348d383a80b5b8bc84312: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 11:44:17.305089 25594 heartbeater.cc:503] Master 127.6.96.62:36835 was elected leader, sending a full tablet report...
I20260926 11:44:17.305622 25613 ts_tablet_manager.cc:1428] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
W20260926 11:44:17.338145 25333 tablet.cc:2369] T 057600b935cb42718995db2672241c78 P 14d13e37020a4a5e851854ea56a6f743: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:44:17.424926 25619 raft_consensus.cc:482] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:44:17.425163 25619 raft_consensus.cc:504] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d13e37020a4a5e851854ea56a6f743" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38499 } } peers { permanent_uuid: "b1d51012b6e348d383a80b5b8bc84312" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 43879 } } peers { permanent_uuid: "b893858014324896bd53816979101126" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 33133 } }
I20260926 11:44:17.426410 25619 leader_election.cc:288] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14d13e37020a4a5e851854ea56a6f743 (127.6.96.1:38499), b1d51012b6e348d383a80b5b8bc84312 (127.6.96.2:43879)
I20260926 11:44:17.431128 25610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:17.427047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:35891 (local address 127.6.96.2:43879)
0926 11:44:17.427152 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:17.427156 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:17.427371 (+ 215us) server_negotiation.cc:407] Connection header received
0926 11:44:17.427450 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:17.427453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:17.427503 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:17.427585 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:17.427798 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.428521 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.429817 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.430432 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.430538 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:17.430598 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:17.430923 (+ 325us) server_negotiation.cc:299] Negotiation successful
0926 11:44:17.431007 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 11:44:17.431128 25607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:17.426771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:47147 (local address 127.6.96.1:38499)
0926 11:44:17.426935 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:17.426939 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:17.426956 (+ 17us) server_negotiation.cc:407] Connection header received
0926 11:44:17.427085 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:17.427089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:17.427143 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:17.427229 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:17.427507 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.428312 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.429810 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.430432 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.430538 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:17.430598 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:17.430928 (+ 330us) server_negotiation.cc:299] Negotiation successful
0926 11:44:17.431000 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 11:44:17.431138 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:17.426660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:38499 (local address 127.6.96.3:47147)
0926 11:44:17.426797 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:17.426875 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:17.426973 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:17.427257 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:17.427260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:17.427280 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:17.427395 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:17.427402 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.428421 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.428424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:17.429668 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:17.429678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.430566 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.430570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:17.430714 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:17.430781 (+ 67us) client_negotiation.cc:769] Sending connection context
0926 11:44:17.430878 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 11:44:17.430973 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":44}
I20260926 11:44:17.431149 25620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:17.426947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:43879 (local address 127.6.96.3:35891)
0926 11:44:17.427272 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:17.427283 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:17.427377 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:17.427595 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:17.427597 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:17.427610 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:17.427701 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:17.427705 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.428614 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.428617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:17.429719 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:17.429726 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.430571 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.430574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:17.430721 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:17.430775 (+ 54us) client_negotiation.cc:769] Sending connection context
0926 11:44:17.430878 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 11:44:17.430973 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":64,"thread_start_us":124,"threads_started":1}
I20260926 11:44:17.431767 25286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "057600b935cb42718995db2672241c78" candidate_uuid: "b893858014324896bd53816979101126" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14d13e37020a4a5e851854ea56a6f743" is_pre_election: true
I20260926 11:44:17.431773 25417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "057600b935cb42718995db2672241c78" candidate_uuid: "b893858014324896bd53816979101126" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d51012b6e348d383a80b5b8bc84312" is_pre_election: true
I20260926 11:44:17.432092 25286 raft_consensus.cc:2454] T 057600b935cb42718995db2672241c78 P 14d13e37020a4a5e851854ea56a6f743 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b893858014324896bd53816979101126 in term 0.
I20260926 11:44:17.432092 25417 raft_consensus.cc:2454] T 057600b935cb42718995db2672241c78 P b1d51012b6e348d383a80b5b8bc84312 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b893858014324896bd53816979101126 in term 0.
I20260926 11:44:17.432581 25483 leader_election.cc:302] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1d51012b6e348d383a80b5b8bc84312, b893858014324896bd53816979101126; no voters:
I20260926 11:44:17.432825 25619 raft_consensus.cc:2781] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:44:17.432912 25619 raft_consensus.cc:482] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:44:17.432956 25619 raft_consensus.cc:3037] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:17.433979 25619 raft_consensus.cc:504] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d13e37020a4a5e851854ea56a6f743" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38499 } } peers { permanent_uuid: "b1d51012b6e348d383a80b5b8bc84312" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 43879 } } peers { permanent_uuid: "b893858014324896bd53816979101126" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 33133 } }
I20260926 11:44:17.434303 25619 leader_election.cc:288] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126 [CANDIDATE]: Term 1 election: Requested vote from peers 14d13e37020a4a5e851854ea56a6f743 (127.6.96.1:38499), b1d51012b6e348d383a80b5b8bc84312 (127.6.96.2:43879)
I20260926 11:44:17.434803 25417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "057600b935cb42718995db2672241c78" candidate_uuid: "b893858014324896bd53816979101126" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d51012b6e348d383a80b5b8bc84312"
I20260926 11:44:17.434798 25286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "057600b935cb42718995db2672241c78" candidate_uuid: "b893858014324896bd53816979101126" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14d13e37020a4a5e851854ea56a6f743"
I20260926 11:44:17.434962 25286 raft_consensus.cc:3037] T 057600b935cb42718995db2672241c78 P 14d13e37020a4a5e851854ea56a6f743 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:17.434962 25417 raft_consensus.cc:3037] T 057600b935cb42718995db2672241c78 P b1d51012b6e348d383a80b5b8bc84312 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:17.436277 25417 raft_consensus.cc:2454] T 057600b935cb42718995db2672241c78 P b1d51012b6e348d383a80b5b8bc84312 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b893858014324896bd53816979101126 in term 1.
I20260926 11:44:17.436277 25286 raft_consensus.cc:2454] T 057600b935cb42718995db2672241c78 P 14d13e37020a4a5e851854ea56a6f743 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b893858014324896bd53816979101126 in term 1.
I20260926 11:44:17.436574 25483 leader_election.cc:302] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14d13e37020a4a5e851854ea56a6f743, b893858014324896bd53816979101126; no voters:
I20260926 11:44:17.436738 25619 raft_consensus.cc:2781] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:17.436985 25619 raft_consensus.cc:686] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126 [term 1 LEADER]: Becoming Leader. State: Replica: b893858014324896bd53816979101126, State: Running, Role: LEADER
I20260926 11:44:17.437297 25619 consensus_queue.cc:237] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d13e37020a4a5e851854ea56a6f743" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38499 } } peers { permanent_uuid: "b1d51012b6e348d383a80b5b8bc84312" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 43879 } } peers { permanent_uuid: "b893858014324896bd53816979101126" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 33133 } }
I20260926 11:44:17.440325 25143 catalog_manager.cc:5495] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126 reported cstate change: term changed from 0 to 1, leader changed from <none> to b893858014324896bd53816979101126 (127.6.96.3). New cstate: current_term: 1 leader_uuid: "b893858014324896bd53816979101126" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d13e37020a4a5e851854ea56a6f743" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d51012b6e348d383a80b5b8bc84312" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 43879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b893858014324896bd53816979101126" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 33133 } health_report { overall_health: HEALTHY } } }
I20260926 11:44:17.470788 25609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:17.467342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39392 (local address 127.6.96.3:33133)
0926 11:44:17.467486 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:17.467490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:17.467500 (+ 10us) server_negotiation.cc:407] Connection header received
0926 11:44:17.467574 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:17.467576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:17.467631 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:17.467723 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:44:17.468001 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.468500 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.469337 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:17.469557 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:17.469644 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:17.469957 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:44:17.470364 (+ 407us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:44:17.470493 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:17.470611 (+ 118us) server_negotiation.cc:299] Negotiation successful
0926 11:44:17.470643 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 11:44:17.476671 25286 raft_consensus.cc:1261] T 057600b935cb42718995db2672241c78 P 14d13e37020a4a5e851854ea56a6f743 [term 1 FOLLOWER]: Refusing update from remote peer b893858014324896bd53816979101126: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:44:17.476742 25417 raft_consensus.cc:1261] T 057600b935cb42718995db2672241c78 P b1d51012b6e348d383a80b5b8bc84312 [term 1 FOLLOWER]: Refusing update from remote peer b893858014324896bd53816979101126: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:44:17.477370 25621 consensus_queue.cc:1035] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14d13e37020a4a5e851854ea56a6f743" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 38499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:44:17.477540 25619 consensus_queue.cc:1035] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d51012b6e348d383a80b5b8bc84312" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 43879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 11:44:17.483460 25595 tablet.cc:2369] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:44:17.483902 25624 mvcc.cc:205] Tried to move back new op lower bound from 7333572843417473024 to 7333572843265732608. Current Snapshot: MvccSnapshot[applied={T|T < 7333572843417473024}]
I20260926 11:44:17.486778 25625 mvcc.cc:205] Tried to move back new op lower bound from 7333572843417473024 to 7333572843265732608. Current Snapshot: MvccSnapshot[applied={T|T < 7333572843417473024}]
I20260926 11:44:17.486934 25626 mvcc.cc:205] Tried to move back new op lower bound from 7333572843417473024 to 7333572843265732608. Current Snapshot: MvccSnapshot[applied={T|T < 7333572843417473024 or (T in {7333572843417473024})}]
I20260926 11:44:17.492543 25142 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:44:17.492736 25142 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:44:17.495435 25142 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 999c6a2262ee49f7a54786a8750c87c2: Sending DeleteTablet for 3 replicas of tablet 057600b935cb42718995db2672241c78
I20260926 11:44:17.496227 25528 tablet_service.cc:1514] Processing DeleteTablet for tablet 057600b935cb42718995db2672241c78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:44:17 UTC) from {username='slave'} at 127.0.0.1:39378
I20260926 11:44:17.496387 25397 tablet_service.cc:1514] Processing DeleteTablet for tablet 057600b935cb42718995db2672241c78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:44:17 UTC) from {username='slave'} at 127.0.0.1:40680
I20260926 11:44:17.497634 25266 tablet_service.cc:1514] Processing DeleteTablet for tablet 057600b935cb42718995db2672241c78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:44:17 UTC) from {username='slave'} at 127.0.0.1:37928
I20260926 11:44:17.497773 25634 tablet_replica.cc:330] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126: stopping tablet replica
I20260926 11:44:17.498143 25634 raft_consensus.cc:2229] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126 [term 1 LEADER]: Raft consensus shutting down.
I20260926 11:44:17.498382 25635 tablet_replica.cc:330] T 057600b935cb42718995db2672241c78 P 14d13e37020a4a5e851854ea56a6f743: stopping tablet replica
I20260926 11:44:17.498498 25634 raft_consensus.cc:2258] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:44:17.498665 25635 raft_consensus.cc:2229] T 057600b935cb42718995db2672241c78 P 14d13e37020a4a5e851854ea56a6f743 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:44:17.499024 25635 raft_consensus.cc:2258] T 057600b935cb42718995db2672241c78 P 14d13e37020a4a5e851854ea56a6f743 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:44:17.500418 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 25204
I20260926 11:44:17.502604 25634 ts_tablet_manager.cc:1905] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:44:17.505364 25634 ts_tablet_manager.cc:1918] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:44:17.505460 25634 log.cc:1196] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/057600b935cb42718995db2672241c78
I20260926 11:44:17.505738 25634 ts_tablet_manager.cc:1939] T 057600b935cb42718995db2672241c78 P b893858014324896bd53816979101126: Deleting consensus metadata
I20260926 11:44:17.507118 25129 catalog_manager.cc:4841] TS b893858014324896bd53816979101126 (127.6.96.3:33133): tablet 057600b935cb42718995db2672241c78 (table test-table [id=348ad8a9485346f39b8d0dc90cab501c]) successfully deleted
W20260926 11:44:17.509135 25130 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.96.1:38499 (error 108)
I20260926 11:44:17.509306 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 25335
I20260926 11:44:17.511085 25606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:17.510664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:38499 (local address 127.0.0.1:37942)
0926 11:44:17.510914 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:17.510990 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.1:38499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
W20260926 11:44:17.511252 25130 catalog_manager.cc:4584] TS 14d13e37020a4a5e851854ea56a6f743 (127.6.96.1:38499): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 057600b935cb42718995db2672241c78: Network error: Client connection negotiation failed: client connection to 127.6.96.1:38499: connect: Connection refused (error 111)
I20260926 11:44:17.517530 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 25466
I20260926 11:44:17.518512 25606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:17.517727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:43879 (local address 127.0.0.1:40690)
0926 11:44:17.518372 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:17.518426 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.2:43879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":565}
I20260926 11:44:17.523195 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 25113
2026-09-26T11:44:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1151 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:44:17 dist-test-slave-xxxx krb5kdc[25639](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:44:17 dist-test-slave-xxxx krb5kdc[25639](info): set up 2 sockets
Sep 26 11:44:17 dist-test-slave-xxxx krb5kdc[25639](info): commencing operation
krb5kdc: starting...
W20260926 11:44:18.474581 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.929s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:44:18 dist-test-slave-xxxx krb5kdc[25639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:44:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:44:18.594657 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:44405
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:44405
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:18.704427 25652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:18.704710 25652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:18.704751 25652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:18.708803 25652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:18.708863 25652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:18.708882 25652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:18.708897 25652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:18.708910 25652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:18.714254 25652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:44405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:44405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:18.715354 25652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:18.716342 25652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:18.725507 25657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:18.725508 25660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:18.731208 25658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:18.731464 25652 server_base.cc:1034] running on GCE node
I20260926 11:44:18.732190 25652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:18.733353 25652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:18.734515 25652 hybrid_clock.cc:648] HybridClock initialized: now 1790423058734486 us; error 41 us; skew 500 ppm
Sep 26 11:44:18 dist-test-slave-xxxx krb5kdc[25639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423058, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:18.744496 25652 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:44:18.745471 25652 webserver.cc:460] Webserver started at http://127.6.96.62:45019/ using document root <none> and password file <none>
I20260926 11:44:18.746032 25652 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:18.746097 25652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:18.746271 25652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:18.748457 25652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "78b48fb8aefd433a917191703b80c396"
format_stamp: "Formatted at 2026-09-26 11:44:18 on dist-test-slave-xxxx"
server_key: "5f0d1608e8fc7ee120395613ede7a68f"
server_key_iv: "2829941411aa75a32c669ee19eb7d85d"
server_key_version: "encryptionkey@0"
I20260926 11:44:18.749030 25652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "78b48fb8aefd433a917191703b80c396"
format_stamp: "Formatted at 2026-09-26 11:44:18 on dist-test-slave-xxxx"
server_key: "5f0d1608e8fc7ee120395613ede7a68f"
server_key_iv: "2829941411aa75a32c669ee19eb7d85d"
server_key_version: "encryptionkey@0"
I20260926 11:44:18.753811 25652 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 11:44:18.756615 25667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:18.758019 25652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260926 11:44:18.758154 25652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "78b48fb8aefd433a917191703b80c396"
format_stamp: "Formatted at 2026-09-26 11:44:18 on dist-test-slave-xxxx"
server_key: "5f0d1608e8fc7ee120395613ede7a68f"
server_key_iv: "2829941411aa75a32c669ee19eb7d85d"
server_key_version: "encryptionkey@0"
I20260926 11:44:18.758263 25652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:18.775928 25652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:18.779448 25652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:18.779662 25652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:18.786581 25652 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:44405
I20260926 11:44:18.786590 25719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:44405 every 8 connection(s)
I20260926 11:44:18.787559 25652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:18.790280 25720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:18.790916 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 25652
I20260926 11:44:18.791047 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:18.791348 6528 external_mini_cluster.cc:1428] Setting key 75273c22c2d654cb0a137c39c7cd8ca5
I20260926 11:44:18.797003 25720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78b48fb8aefd433a917191703b80c396: Bootstrap starting.
I20260926 11:44:18.799625 25720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78b48fb8aefd433a917191703b80c396: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:18.800418 25720 log.cc:824] T 00000000000000000000000000000000 P 78b48fb8aefd433a917191703b80c396: Log is configured to *not* fsync() on all Append() calls
Sep 26 11:44:18 dist-test-slave-xxxx krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:44:18.802395 25720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78b48fb8aefd433a917191703b80c396: No bootstrap required, opened a new log
I20260926 11:44:18.805549 25723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:18.792683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49170 (local address 127.6.96.62:44405)
0926 11:44:18.793100 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:18.793120 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:18.793261 (+ 141us) server_negotiation.cc:407] Connection header received
0926 11:44:18.794128 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:18.794152 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:18.794598 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:18.795026 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:18.795663 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:18.797135 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:18.798002 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:18.798672 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:18.798854 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:18.801350 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:18.801356 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:18.801370 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:18.801398 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:18.802989 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:18.803375 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:18.803381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:18.803387 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:18.803491 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:18.803734 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:18.803737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:18.803739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:18.804125 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:18.804294 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:18.804579 (+ 285us) server_negotiation.cc:299] Negotiation successful
0926 11:44:18.804857 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":104,"threads_started":1}
I20260926 11:44:18.805979 25720 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78b48fb8aefd433a917191703b80c396 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b48fb8aefd433a917191703b80c396" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44405 } }
I20260926 11:44:18.806209 25720 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78b48fb8aefd433a917191703b80c396 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:18.806273 25720 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78b48fb8aefd433a917191703b80c396 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78b48fb8aefd433a917191703b80c396, State: Initialized, Role: FOLLOWER
I20260926 11:44:18.806725 25720 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78b48fb8aefd433a917191703b80c396 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b48fb8aefd433a917191703b80c396" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44405 } }
I20260926 11:44:18.806936 25720 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78b48fb8aefd433a917191703b80c396 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:44:18.807001 25720 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78b48fb8aefd433a917191703b80c396 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:44:18.807094 25720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78b48fb8aefd433a917191703b80c396 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:18.808195 25720 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78b48fb8aefd433a917191703b80c396 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b48fb8aefd433a917191703b80c396" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44405 } }
I20260926 11:44:18.808491 25720 leader_election.cc:302] T 00000000000000000000000000000000 P 78b48fb8aefd433a917191703b80c396 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78b48fb8aefd433a917191703b80c396; no voters:
I20260926 11:44:18.808764 25720 leader_election.cc:288] T 00000000000000000000000000000000 P 78b48fb8aefd433a917191703b80c396 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:44:18.808885 25725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78b48fb8aefd433a917191703b80c396 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:18.809087 25725 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78b48fb8aefd433a917191703b80c396 [term 1 LEADER]: Becoming Leader. State: Replica: 78b48fb8aefd433a917191703b80c396, State: Running, Role: LEADER
I20260926 11:44:18.809378 25725 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78b48fb8aefd433a917191703b80c396 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b48fb8aefd433a917191703b80c396" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44405 } }
I20260926 11:44:18.809878 25720 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78b48fb8aefd433a917191703b80c396 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:18.810817 25726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78b48fb8aefd433a917191703b80c396 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78b48fb8aefd433a917191703b80c396" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b48fb8aefd433a917191703b80c396" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44405 } } }
I20260926 11:44:18.810966 25726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78b48fb8aefd433a917191703b80c396 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:18.811240 25727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78b48fb8aefd433a917191703b80c396 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78b48fb8aefd433a917191703b80c396. Latest consensus state: current_term: 1 leader_uuid: "78b48fb8aefd433a917191703b80c396" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b48fb8aefd433a917191703b80c396" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 44405 } } }
I20260926 11:44:18.811333 25727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78b48fb8aefd433a917191703b80c396 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:18.811482 25734 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:18.814689 25734 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:18.819859 25734 catalog_manager.cc:1347] Generated new cluster ID: e11ce992d39c45a8a90721cf089c285a
I20260926 11:44:18.819923 25734 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:18.839555 25734 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:18.840524 25734 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:18.852029 25734 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78b48fb8aefd433a917191703b80c396: Generated new TSK 0
I20260926 11:44:18.852873 25734 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.96.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
Entry for principal kudu/127.6.96.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab.
I20260926 11:44:18.889740 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:44405
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:19.002735 25746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:19.003043 25746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:19.003103 25746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:19.007129 25746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:19.007197 25746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:19.007277 25746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:44:19.012801 25746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.1.keytab
--principal=kudu/127.6.96.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:44405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:19.013891 25746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:19.014969 25746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:19.024441 25751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:19.024497 25752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:19.024510 25754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:19.024596 25746 server_base.cc:1034] running on GCE node
I20260926 11:44:19.025056 25746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:19.025684 25746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:19.026859 25746 hybrid_clock.cc:648] HybridClock initialized: now 1790423059026835 us; error 41 us; skew 500 ppm
Sep 26 11:44:19 dist-test-slave-xxxx krb5kdc[25639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423059, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:19.029429 25746 init.cc:377] Logged in from keytab as kudu/127.6.96.1@KRBTEST.COM (short username kudu)
I20260926 11:44:19.030283 25746 webserver.cc:460] Webserver started at http://127.6.96.1:44293/ using document root <none> and password file <none>
I20260926 11:44:19.030808 25746 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:19.030912 25746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:19.031129 25746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:19.033108 25746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "83991525f5ad4c6c93cac4dc521832a8"
format_stamp: "Formatted at 2026-09-26 11:44:19 on dist-test-slave-xxxx"
server_key: "c55b7a3099f70eaca48386960e1da9d9"
server_key_iv: "4b371eccb548d43a2f0292448477b635"
server_key_version: "encryptionkey@0"
I20260926 11:44:19.033635 25746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "83991525f5ad4c6c93cac4dc521832a8"
format_stamp: "Formatted at 2026-09-26 11:44:19 on dist-test-slave-xxxx"
server_key: "c55b7a3099f70eaca48386960e1da9d9"
server_key_iv: "4b371eccb548d43a2f0292448477b635"
server_key_version: "encryptionkey@0"
I20260926 11:44:19.037209 25746 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 11:44:19.039553 25761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:19.040699 25746 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:44:19.040803 25746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "83991525f5ad4c6c93cac4dc521832a8"
format_stamp: "Formatted at 2026-09-26 11:44:19 on dist-test-slave-xxxx"
server_key: "c55b7a3099f70eaca48386960e1da9d9"
server_key_iv: "4b371eccb548d43a2f0292448477b635"
server_key_version: "encryptionkey@0"
I20260926 11:44:19.040896 25746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:19.052223 25746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:19.055051 25746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:19.055218 25746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:19.055770 25746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:44:19.056578 25746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:44:19.056623 25746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:19.056675 25746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:44:19.056704 25746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:19.065358 25746 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:33049
I20260926 11:44:19.065416 25874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:33049 every 8 connection(s)
I20260926 11:44:19.066256 25746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
Sep 26 11:44:19 dist-test-slave-xxxx krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423059, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.1@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:44:19.076265 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 25746
I20260926 11:44:19.076416 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:44:19.076685 6528 external_mini_cluster.cc:1428] Setting key ef71501ab3dd24868ea9acbc243783f3
I20260926 11:44:19.080451 25723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:19.068052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:58681 (local address 127.6.96.62:44405)
0926 11:44:19.068188 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:19.068193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:19.069057 (+ 864us) server_negotiation.cc:407] Connection header received
0926 11:44:19.070303 (+ 1246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:19.070307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:19.070358 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:19.070447 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:19.071768 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.072321 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.073465 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.073713 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.073804 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:19.076515 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:19.076519 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:19.076522 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:19.076548 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:19.077972 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:19.078557 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:19.078559 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:19.078560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:19.078636 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:19.079100 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:19.079103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:19.079104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:19.079311 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:19.079397 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:19.080102 (+ 705us) server_negotiation.cc:299] Negotiation successful
0926 11:44:19.080283 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 11:44:19.081176 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:19.068470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44405 (local address 127.6.96.1:58681)
0926 11:44:19.068910 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:19.068953 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:19.070072 (+ 1119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:19.070636 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:19.070644 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:19.071083 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:19.071596 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:19.071610 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.072431 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.072433 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:19.073334 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:19.073342 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.073854 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.073856 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:19.074006 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:19.074572 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:19.074589 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:19.076271 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:19.078099 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:19.078106 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:19.078124 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:19.078466 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:19.078720 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:19.078723 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:19.078725 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:19.078977 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:19.079410 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:19.079417 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:19.079782 (+ 365us) client_negotiation.cc:769] Sending connection context
0926 11:44:19.079999 (+ 217us) client_negotiation.cc:240] Negotiation successful
0926 11:44:19.080286 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":106,"threads_started":1}
I20260926 11:44:19.082693 25875 heartbeater.cc:347] Connected to a master server at 127.6.96.62:44405
I20260926 11:44:19.082993 25875 heartbeater.cc:464] Registering TS with master...
I20260926 11:44:19.083600 25875 heartbeater.cc:511] Master 127.6.96.62:44405 requested a full tablet report, sending...
I20260926 11:44:19.085206 25684 ts_manager.cc:194] Registered new tserver with Master: 83991525f5ad4c6c93cac4dc521832a8 (127.6.96.1:33049)
WARNING: no policy specified for kudu/127.6.96.2@KRBTEST.COM; defaulting to no policy
I20260926 11:44:19.086404 25684 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.1@KRBTEST.COM'} at 127.6.96.1:58681
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
Entry for principal kudu/127.6.96.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab.
I20260926 11:44:19.106307 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:44405
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:19.214116 25880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:19.214494 25880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:19.214558 25880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:19.218700 25880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:19.218788 25880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:19.218891 25880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:44:19.224545 25880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.2.keytab
--principal=kudu/127.6.96.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:44405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:19.225615 25880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:19.226649 25880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:19.235930 25886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:19.235896 25888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:19.236078 25880 server_base.cc:1034] running on GCE node
W20260926 11:44:19.235958 25885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:19.236512 25880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:19.237155 25880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:19.238296 25880 hybrid_clock.cc:648] HybridClock initialized: now 1790423059238278 us; error 46 us; skew 500 ppm
Sep 26 11:44:19 dist-test-slave-xxxx krb5kdc[25639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423059, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:19.240960 25880 init.cc:377] Logged in from keytab as kudu/127.6.96.2@KRBTEST.COM (short username kudu)
I20260926 11:44:19.241827 25880 webserver.cc:460] Webserver started at http://127.6.96.2:46627/ using document root <none> and password file <none>
I20260926 11:44:19.242350 25880 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:19.242408 25880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:19.242597 25880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:19.244748 25880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "6a9d9cf5a1a5422fb46fcc5e2eadbf4c"
format_stamp: "Formatted at 2026-09-26 11:44:19 on dist-test-slave-xxxx"
server_key: "af010d90841d4ee7b6a99a5e3574b020"
server_key_iv: "cd799ef73f50fe31de957a34543532d3"
server_key_version: "encryptionkey@0"
I20260926 11:44:19.245275 25880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "6a9d9cf5a1a5422fb46fcc5e2eadbf4c"
format_stamp: "Formatted at 2026-09-26 11:44:19 on dist-test-slave-xxxx"
server_key: "af010d90841d4ee7b6a99a5e3574b020"
server_key_iv: "cd799ef73f50fe31de957a34543532d3"
server_key_version: "encryptionkey@0"
I20260926 11:44:19.248921 25880 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 11:44:19.251278 25895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:19.252539 25880 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:44:19.252652 25880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "6a9d9cf5a1a5422fb46fcc5e2eadbf4c"
format_stamp: "Formatted at 2026-09-26 11:44:19 on dist-test-slave-xxxx"
server_key: "af010d90841d4ee7b6a99a5e3574b020"
server_key_iv: "cd799ef73f50fe31de957a34543532d3"
server_key_version: "encryptionkey@0"
I20260926 11:44:19.252759 25880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:19.278484 25880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:19.281993 25880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:19.282204 25880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:19.282872 25880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:44:19.283924 25880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:44:19.283993 25880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:19.284054 25880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:44:19.284082 25880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:19.293063 25880 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:37577
I20260926 11:44:19.293134 26008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:37577 every 8 connection(s)
I20260926 11:44:19.294040 25880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:44:19.302580 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 25880
I20260926 11:44:19.302733 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:44:19.303072 6528 external_mini_cluster.cc:1428] Setting key 852b27baae3764cd9c83b0741f5e9a0a
Sep 26 11:44:19 dist-test-slave-xxxx krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423059, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.2@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:44:19.308969 25723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:19.296024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:39961 (local address 127.6.96.62:44405)
0926 11:44:19.296153 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:19.296156 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:19.297031 (+ 875us) server_negotiation.cc:407] Connection header received
0926 11:44:19.298319 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:19.298322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:19.298382 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:19.298455 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:19.299811 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.300475 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.301762 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.302001 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.302092 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:19.305042 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:19.305046 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:19.305049 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:19.305079 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:19.306519 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:19.307180 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:19.307183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:19.307184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:19.307262 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:19.307723 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:19.307727 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:19.307728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:19.307936 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:19.308026 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:19.308659 (+ 633us) server_negotiation.cc:299] Negotiation successful
0926 11:44:19.308772 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 11:44:19.310138 26011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:19.296395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44405 (local address 127.6.96.2:39961)
0926 11:44:19.296870 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:19.296916 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:19.298078 (+ 1162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:19.298645 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:19.298653 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:19.299163 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:19.299660 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:19.299674 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.300589 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.300592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:19.301639 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:19.301649 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.302149 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.302152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:19.302375 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:19.303079 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:19.303101 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:19.304840 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:19.306662 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:19.306669 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:19.306687 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:19.307081 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:19.307359 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:19.307362 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:19.307365 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:19.307571 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:19.308048 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:19.308055 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:19.308385 (+ 330us) client_negotiation.cc:769] Sending connection context
0926 11:44:19.308597 (+ 212us) client_negotiation.cc:240] Negotiation successful
0926 11:44:19.308974 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":113,"threads_started":1}
I20260926 11:44:19.311439 26009 heartbeater.cc:347] Connected to a master server at 127.6.96.62:44405
I20260926 11:44:19.311730 26009 heartbeater.cc:464] Registering TS with master...
I20260926 11:44:19.312337 26009 heartbeater.cc:511] Master 127.6.96.62:44405 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.96.3@KRBTEST.COM; defaulting to no policy
I20260926 11:44:19.313414 25684 ts_manager.cc:194] Registered new tserver with Master: 6a9d9cf5a1a5422fb46fcc5e2eadbf4c (127.6.96.2:37577)
I20260926 11:44:19.313971 25684 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.2@KRBTEST.COM'} at 127.6.96.2:39961
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
Entry for principal kudu/127.6.96.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab.
I20260926 11:44:19.330137 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:44405
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:19.441277 26014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:19.441560 26014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:19.441603 26014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:19.445684 26014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:19.445750 26014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:19.445809 26014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:44:19.451212 26014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.3.keytab
--principal=kudu/127.6.96.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:44405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:19.452237 26014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:19.453205 26014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:19.462421 26022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:19.462446 26019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:19.462616 26014 server_base.cc:1034] running on GCE node
W20260926 11:44:19.462783 26020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:19.463228 26014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:19.463855 26014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:19.465024 26014 hybrid_clock.cc:648] HybridClock initialized: now 1790423059465009 us; error 30 us; skew 500 ppm
Sep 26 11:44:19 dist-test-slave-xxxx krb5kdc[25639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423059, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:19.467732 26014 init.cc:377] Logged in from keytab as kudu/127.6.96.3@KRBTEST.COM (short username kudu)
I20260926 11:44:19.468629 26014 webserver.cc:460] Webserver started at http://127.6.96.3:40103/ using document root <none> and password file <none>
I20260926 11:44:19.469184 26014 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:19.469250 26014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:19.469455 26014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:19.471549 26014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "36021c39e44e48969a03206741895e19"
format_stamp: "Formatted at 2026-09-26 11:44:19 on dist-test-slave-xxxx"
server_key: "36eaf6590efa455f74d68e4947fb9bb6"
server_key_iv: "dceaef4fcf5eef2f01bb063f6dff1fb6"
server_key_version: "encryptionkey@0"
I20260926 11:44:19.472086 26014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "36021c39e44e48969a03206741895e19"
format_stamp: "Formatted at 2026-09-26 11:44:19 on dist-test-slave-xxxx"
server_key: "36eaf6590efa455f74d68e4947fb9bb6"
server_key_iv: "dceaef4fcf5eef2f01bb063f6dff1fb6"
server_key_version: "encryptionkey@0"
I20260926 11:44:19.475764 26014 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 11:44:19.478149 26029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:19.479382 26014 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 11:44:19.479497 26014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "36021c39e44e48969a03206741895e19"
format_stamp: "Formatted at 2026-09-26 11:44:19 on dist-test-slave-xxxx"
server_key: "36eaf6590efa455f74d68e4947fb9bb6"
server_key_iv: "dceaef4fcf5eef2f01bb063f6dff1fb6"
server_key_version: "encryptionkey@0"
I20260926 11:44:19.479594 26014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:19.492429 26014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:19.496302 26014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:19.496487 26014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:19.497195 26014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:44:19.498241 26014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:44:19.498306 26014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:19.498360 26014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:44:19.498390 26014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:19.507355 26014 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:37155
I20260926 11:44:19.507377 26142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:37155 every 8 connection(s)
I20260926 11:44:19.508270 26014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:44:19.516315 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 26014
I20260926 11:44:19.516476 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:44:19.516808 6528 external_mini_cluster.cc:1428] Setting key 1cc0dc7324d06f755efca4636dd1b19c
Sep 26 11:44:19 dist-test-slave-xxxx krb5kdc[25639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423059, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.3@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:44:19.522266 25723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:19.510176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:60027 (local address 127.6.96.62:44405)
0926 11:44:19.510326 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:19.510329 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:19.511304 (+ 975us) server_negotiation.cc:407] Connection header received
0926 11:44:19.512677 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:19.512680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:19.512732 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:19.512821 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:19.514051 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.514602 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.515814 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.516069 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.516157 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:19.519078 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:19.519082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:19.519085 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:19.519114 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:19.520325 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:19.520831 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:19.520833 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:19.520835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:19.520905 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:19.521252 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:19.521255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:19.521257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:19.521505 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:19.521586 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:19.522036 (+ 450us) server_negotiation.cc:299] Negotiation successful
0926 11:44:19.522144 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 11:44:19.523249 26145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:19.510558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:44405 (local address 127.6.96.3:60027)
0926 11:44:19.511119 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:19.511196 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:19.512443 (+ 1247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:19.513016 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:19.513024 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:19.513404 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:19.513903 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:19.513917 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.514715 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.514718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:19.515685 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:19.515703 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.516231 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.516236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:19.516447 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:19.517169 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:19.517186 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:19.518937 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:19.520429 (+ 1492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:19.520434 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:19.520447 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:19.520748 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:19.520972 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:19.520975 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:19.520977 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:19.521123 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:19.521585 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:19.521590 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:19.521835 (+ 245us) client_negotiation.cc:769] Sending connection context
0926 11:44:19.521992 (+ 157us) client_negotiation.cc:240] Negotiation successful
0926 11:44:19.522327 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":133,"threads_started":1}
I20260926 11:44:19.524322 26143 heartbeater.cc:347] Connected to a master server at 127.6.96.62:44405
I20260926 11:44:19.524545 26143 heartbeater.cc:464] Registering TS with master...
I20260926 11:44:19.525038 26143 heartbeater.cc:511] Master 127.6.96.62:44405 requested a full tablet report, sending...
I20260926 11:44:19.525981 25684 ts_manager.cc:194] Registered new tserver with Master: 36021c39e44e48969a03206741895e19 (127.6.96.3:37155)
I20260926 11:44:19.526520 25684 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.96.3@KRBTEST.COM'} at 127.6.96.3:60027
I20260926 11:44:19.530627 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 11:44:19.539615 25723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:19.532629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49180 (local address 127.6.96.62:44405)
0926 11:44:19.532743 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:19.532747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:19.532954 (+ 207us) server_negotiation.cc:407] Connection header received
0926 11:44:19.533079 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:19.533082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:19.533122 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:19.533181 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:19.533870 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.534390 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.535255 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.535493 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.535580 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:19.536906 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:19.536910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:19.536913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:19.536940 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:19.537987 (+ 1047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:19.538364 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:19.538367 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:19.538369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:19.538446 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:19.538681 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:19.538684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:19.538686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:19.538967 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:19.539112 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:19.539340 (+ 228us) server_negotiation.cc:299] Negotiation successful
0926 11:44:19.539460 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 11:44:19.542555 25684 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 11:44:19.545799 25684 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:44:19.561110 26157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:19.554495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:37155 (local address 127.0.0.1:41566)
0926 11:44:19.555712 (+ 1217us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:19.555725 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:19.555820 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:19.556584 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:19.556587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:19.556613 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:19.556814 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:19.556822 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.558197 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.558200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:19.559392 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:19.559402 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.560530 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.560534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:19.560654 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:19.560692 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 11:44:19.560809 (+ 117us) client_negotiation.cc:240] Negotiation successful
0926 11:44:19.560942 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"mutex_wait_us":5,"spinlock_wait_cycles":768,"thread_start_us":92,"threads_started":1}
I20260926 11:44:19.561128 26159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:19.555322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41566 (local address 127.6.96.3:37155)
0926 11:44:19.555782 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:19.555794 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:19.555817 (+ 23us) server_negotiation.cc:407] Connection header received
0926 11:44:19.555970 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:19.555977 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:19.556249 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:19.556386 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:19.556934 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.558039 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.559529 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.560402 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.560522 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:19.560687 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:19.560850 (+ 163us) server_negotiation.cc:299] Negotiation successful
0926 11:44:19.560959 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":149,"threads_started":1}
I20260926 11:44:19.561409 26156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:19.554610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46788 (local address 127.6.96.2:37577)
0926 11:44:19.556006 (+ 1396us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:19.556017 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:19.556071 (+ 54us) server_negotiation.cc:407] Connection header received
0926 11:44:19.556139 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:19.556143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:19.556295 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:19.556417 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:19.556729 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.558003 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.559911 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.560495 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.560597 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:19.560745 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:19.561170 (+ 425us) server_negotiation.cc:299] Negotiation successful
0926 11:44:19.561272 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1299,"thread_start_us":132,"threads_started":1}
I20260926 11:44:19.561514 26158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:19.555074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38170 (local address 127.6.96.1:33049)
0926 11:44:19.556041 (+ 967us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:19.556057 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:19.556086 (+ 29us) server_negotiation.cc:407] Connection header received
0926 11:44:19.556163 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:19.556168 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:19.556319 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:19.556449 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:19.556967 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.557817 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.559641 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.560464 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.560560 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:19.560687 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:19.561248 (+ 561us) server_negotiation.cc:299] Negotiation successful
0926 11:44:19.561359 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":854,"thread_start_us":69,"threads_started":1}
I20260926 11:44:19.561731 26154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:19.554432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:37577 (local address 127.0.0.1:46788)
0926 11:44:19.555323 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:19.555377 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:19.555524 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:19.556447 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:19.556453 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:19.556474 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:19.556610 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:19.556617 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.558141 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.558144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:19.559788 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:19.559797 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.560816 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.560819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:19.560935 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:19.560954 (+ 19us) client_negotiation.cc:769] Sending connection context
0926 11:44:19.561013 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 11:44:19.561076 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"mutex_wait_us":564,"thread_start_us":109,"threads_started":1}
I20260926 11:44:19.562004 26155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:19.554789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:33049 (local address 127.0.0.1:38170)
0926 11:44:19.555485 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:19.555497 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:19.555563 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:19.556584 (+ 1021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:19.556587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:19.556619 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:19.556825 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:19.556831 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.557996 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.558000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:19.559502 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:19.559513 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.560750 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.560753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:19.560865 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:19.560910 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 11:44:19.560982 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 11:44:19.561058 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"mutex_wait_us":120,"spinlock_wait_cycles":9728,"thread_start_us":265,"threads_started":1}
I20260926 11:44:19.564747 26077 tablet_service.cc:1467] Processing CreateTablet for tablet 23d53c56f4fb4b13a3e2c5f2bdd0a94a (DEFAULT_TABLE table=test-table [id=78f88a93848243eab5d4717e0dfeed41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:44:19.564828 25809 tablet_service.cc:1467] Processing CreateTablet for tablet 23d53c56f4fb4b13a3e2c5f2bdd0a94a (DEFAULT_TABLE table=test-table [id=78f88a93848243eab5d4717e0dfeed41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:44:19.564829 25943 tablet_service.cc:1467] Processing CreateTablet for tablet 23d53c56f4fb4b13a3e2c5f2bdd0a94a (DEFAULT_TABLE table=test-table [id=78f88a93848243eab5d4717e0dfeed41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:44:19.565769 26077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23d53c56f4fb4b13a3e2c5f2bdd0a94a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:19.565842 25809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23d53c56f4fb4b13a3e2c5f2bdd0a94a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:19.565842 25943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23d53c56f4fb4b13a3e2c5f2bdd0a94a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:19.572153 26160 tablet_bootstrap.cc:492] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 36021c39e44e48969a03206741895e19: Bootstrap starting.
I20260926 11:44:19.573557 26162 tablet_bootstrap.cc:492] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 83991525f5ad4c6c93cac4dc521832a8: Bootstrap starting.
I20260926 11:44:19.573568 26161 tablet_bootstrap.cc:492] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c: Bootstrap starting.
I20260926 11:44:19.574528 26160 tablet_bootstrap.cc:654] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 36021c39e44e48969a03206741895e19: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:19.575282 26162 tablet_bootstrap.cc:654] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 83991525f5ad4c6c93cac4dc521832a8: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:19.575631 26161 tablet_bootstrap.cc:654] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:19.575770 26160 log.cc:824] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 36021c39e44e48969a03206741895e19: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:19.576236 26162 log.cc:824] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 83991525f5ad4c6c93cac4dc521832a8: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:19.576715 26161 log.cc:824] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:19.577962 26160 tablet_bootstrap.cc:492] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 36021c39e44e48969a03206741895e19: No bootstrap required, opened a new log
I20260926 11:44:19.578155 26160 ts_tablet_manager.cc:1397] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 36021c39e44e48969a03206741895e19: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260926 11:44:19.578181 26162 tablet_bootstrap.cc:492] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 83991525f5ad4c6c93cac4dc521832a8: No bootstrap required, opened a new log
I20260926 11:44:19.578336 26162 ts_tablet_manager.cc:1397] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 83991525f5ad4c6c93cac4dc521832a8: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 11:44:19.578534 26161 tablet_bootstrap.cc:492] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c: No bootstrap required, opened a new log
I20260926 11:44:19.578701 26161 ts_tablet_manager.cc:1397] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 11:44:19.581167 26162 raft_consensus.cc:348] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 83991525f5ad4c6c93cac4dc521832a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9d9cf5a1a5422fb46fcc5e2eadbf4c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 37577 } } peers { permanent_uuid: "36021c39e44e48969a03206741895e19" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37155 } } peers { permanent_uuid: "83991525f5ad4c6c93cac4dc521832a8" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 33049 } }
I20260926 11:44:19.581390 26162 raft_consensus.cc:374] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 83991525f5ad4c6c93cac4dc521832a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:19.581458 26162 raft_consensus.cc:729] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 83991525f5ad4c6c93cac4dc521832a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83991525f5ad4c6c93cac4dc521832a8, State: Initialized, Role: FOLLOWER
I20260926 11:44:19.581933 26162 consensus_queue.cc:260] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 83991525f5ad4c6c93cac4dc521832a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9d9cf5a1a5422fb46fcc5e2eadbf4c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 37577 } } peers { permanent_uuid: "36021c39e44e48969a03206741895e19" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37155 } } peers { permanent_uuid: "83991525f5ad4c6c93cac4dc521832a8" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 33049 } }
I20260926 11:44:19.581951 26160 raft_consensus.cc:348] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 36021c39e44e48969a03206741895e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9d9cf5a1a5422fb46fcc5e2eadbf4c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 37577 } } peers { permanent_uuid: "36021c39e44e48969a03206741895e19" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37155 } } peers { permanent_uuid: "83991525f5ad4c6c93cac4dc521832a8" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 33049 } }
I20260926 11:44:19.582214 26160 raft_consensus.cc:374] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 36021c39e44e48969a03206741895e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:19.582289 26160 raft_consensus.cc:729] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 36021c39e44e48969a03206741895e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36021c39e44e48969a03206741895e19, State: Initialized, Role: FOLLOWER
I20260926 11:44:19.582259 26161 raft_consensus.cc:348] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9d9cf5a1a5422fb46fcc5e2eadbf4c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 37577 } } peers { permanent_uuid: "36021c39e44e48969a03206741895e19" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37155 } } peers { permanent_uuid: "83991525f5ad4c6c93cac4dc521832a8" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 33049 } }
I20260926 11:44:19.582508 26161 raft_consensus.cc:374] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:19.582561 26161 raft_consensus.cc:729] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a9d9cf5a1a5422fb46fcc5e2eadbf4c, State: Initialized, Role: FOLLOWER
I20260926 11:44:19.582733 25875 heartbeater.cc:503] Master 127.6.96.62:44405 was elected leader, sending a full tablet report...
I20260926 11:44:19.582752 26160 consensus_queue.cc:260] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 36021c39e44e48969a03206741895e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9d9cf5a1a5422fb46fcc5e2eadbf4c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 37577 } } peers { permanent_uuid: "36021c39e44e48969a03206741895e19" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37155 } } peers { permanent_uuid: "83991525f5ad4c6c93cac4dc521832a8" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 33049 } }
I20260926 11:44:19.582959 26161 consensus_queue.cc:260] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9d9cf5a1a5422fb46fcc5e2eadbf4c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 37577 } } peers { permanent_uuid: "36021c39e44e48969a03206741895e19" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37155 } } peers { permanent_uuid: "83991525f5ad4c6c93cac4dc521832a8" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 33049 } }
I20260926 11:44:19.583065 26162 ts_tablet_manager.cc:1428] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 83991525f5ad4c6c93cac4dc521832a8: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 11:44:19.583783 26161 ts_tablet_manager.cc:1428] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260926 11:44:19.583811 26160 ts_tablet_manager.cc:1428] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 36021c39e44e48969a03206741895e19: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 11:44:19.584123 26009 heartbeater.cc:503] Master 127.6.96.62:44405 was elected leader, sending a full tablet report...
I20260926 11:44:19.584914 26143 heartbeater.cc:503] Master 127.6.96.62:44405 was elected leader, sending a full tablet report...
W20260926 11:44:19.759244 26144 tablet.cc:2369] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 36021c39e44e48969a03206741895e19: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 11:44:19.795183 26010 tablet.cc:2369] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 11:44:19.817576 25876 tablet.cc:2369] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 83991525f5ad4c6c93cac4dc521832a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:44:19.867025 26168 raft_consensus.cc:482] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:44:19.867233 26168 raft_consensus.cc:504] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9d9cf5a1a5422fb46fcc5e2eadbf4c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 37577 } } peers { permanent_uuid: "36021c39e44e48969a03206741895e19" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37155 } } peers { permanent_uuid: "83991525f5ad4c6c93cac4dc521832a8" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 33049 } }
I20260926 11:44:19.868403 26168 leader_election.cc:288] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36021c39e44e48969a03206741895e19 (127.6.96.3:37155), 83991525f5ad4c6c93cac4dc521832a8 (127.6.96.1:33049)
I20260926 11:44:19.873536 26159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:19.868750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:53323 (local address 127.6.96.3:37155)
0926 11:44:19.868942 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:19.868947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:19.869017 (+ 70us) server_negotiation.cc:407] Connection header received
0926 11:44:19.869214 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:19.869217 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:19.869260 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:19.869340 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:19.869687 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.870476 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.872314 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.872918 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.873026 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:19.873091 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:19.873336 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 11:44:19.873404 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260926 11:44:19.873536 26158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:19.868768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:45943 (local address 127.6.96.1:33049)
0926 11:44:19.868893 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:19.868898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:19.869065 (+ 167us) server_negotiation.cc:407] Connection header received
0926 11:44:19.869200 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:19.869203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:19.869254 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:19.869340 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:19.869687 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.870481 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.872319 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.872918 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.873026 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:19.873091 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:19.873336 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 11:44:19.873404 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 11:44:19.873581 26170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:19.868642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:37155 (local address 127.6.96.2:53323)
0926 11:44:19.868921 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:19.868937 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:19.869104 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:19.869360 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:19.869362 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:19.869387 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:19.869593 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:19.869599 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.870596 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.870600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:19.872191 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:19.872199 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.873032 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.873036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:19.873215 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:19.873249 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 11:44:19.873312 (+ 63us) client_negotiation.cc:240] Negotiation successful
0926 11:44:19.873405 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":265,"spinlock_wait_cycles":32000,"thread_start_us":87,"threads_started":1}
I20260926 11:44:19.873589 26169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:19.868650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:33049 (local address 127.6.96.2:45943)
0926 11:44:19.868989 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:19.869000 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:19.869090 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:19.869359 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:19.869362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:19.869389 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:19.869584 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:19.869591 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.870596 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.870600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:19.872187 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:19.872199 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.873032 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.873036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:19.873193 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:19.873238 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 11:44:19.873312 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 11:44:19.873408 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":210,"spinlock_wait_cycles":7936,"thread_start_us":73,"threads_started":1}
I20260926 11:44:19.874310 25829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23d53c56f4fb4b13a3e2c5f2bdd0a94a" candidate_uuid: "6a9d9cf5a1a5422fb46fcc5e2eadbf4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83991525f5ad4c6c93cac4dc521832a8" is_pre_election: true
I20260926 11:44:19.874380 26097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23d53c56f4fb4b13a3e2c5f2bdd0a94a" candidate_uuid: "6a9d9cf5a1a5422fb46fcc5e2eadbf4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36021c39e44e48969a03206741895e19" is_pre_election: true
I20260926 11:44:19.874619 25829 raft_consensus.cc:2454] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 83991525f5ad4c6c93cac4dc521832a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a9d9cf5a1a5422fb46fcc5e2eadbf4c in term 0.
I20260926 11:44:19.874658 26097 raft_consensus.cc:2454] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 36021c39e44e48969a03206741895e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a9d9cf5a1a5422fb46fcc5e2eadbf4c in term 0.
I20260926 11:44:19.875092 25899 leader_election.cc:302] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a9d9cf5a1a5422fb46fcc5e2eadbf4c, 83991525f5ad4c6c93cac4dc521832a8; no voters:
I20260926 11:44:19.875345 26168 raft_consensus.cc:2781] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:44:19.875432 26168 raft_consensus.cc:482] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:44:19.875483 26168 raft_consensus.cc:3037] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:19.876511 26168 raft_consensus.cc:504] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9d9cf5a1a5422fb46fcc5e2eadbf4c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 37577 } } peers { permanent_uuid: "36021c39e44e48969a03206741895e19" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37155 } } peers { permanent_uuid: "83991525f5ad4c6c93cac4dc521832a8" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 33049 } }
I20260926 11:44:19.876821 26168 leader_election.cc:288] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c [CANDIDATE]: Term 1 election: Requested vote from peers 36021c39e44e48969a03206741895e19 (127.6.96.3:37155), 83991525f5ad4c6c93cac4dc521832a8 (127.6.96.1:33049)
I20260926 11:44:19.877162 26097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23d53c56f4fb4b13a3e2c5f2bdd0a94a" candidate_uuid: "6a9d9cf5a1a5422fb46fcc5e2eadbf4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36021c39e44e48969a03206741895e19"
I20260926 11:44:19.877194 25829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23d53c56f4fb4b13a3e2c5f2bdd0a94a" candidate_uuid: "6a9d9cf5a1a5422fb46fcc5e2eadbf4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83991525f5ad4c6c93cac4dc521832a8"
I20260926 11:44:19.877281 26097 raft_consensus.cc:3037] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 36021c39e44e48969a03206741895e19 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:19.877300 25829 raft_consensus.cc:3037] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 83991525f5ad4c6c93cac4dc521832a8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:19.878273 25829 raft_consensus.cc:2454] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 83991525f5ad4c6c93cac4dc521832a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a9d9cf5a1a5422fb46fcc5e2eadbf4c in term 1.
I20260926 11:44:19.878357 26097 raft_consensus.cc:2454] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 36021c39e44e48969a03206741895e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a9d9cf5a1a5422fb46fcc5e2eadbf4c in term 1.
I20260926 11:44:19.878557 25899 leader_election.cc:302] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a9d9cf5a1a5422fb46fcc5e2eadbf4c, 83991525f5ad4c6c93cac4dc521832a8; no voters:
I20260926 11:44:19.878717 26168 raft_consensus.cc:2781] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:19.879001 26168 raft_consensus.cc:686] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c [term 1 LEADER]: Becoming Leader. State: Replica: 6a9d9cf5a1a5422fb46fcc5e2eadbf4c, State: Running, Role: LEADER
I20260926 11:44:19.879359 26168 consensus_queue.cc:237] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9d9cf5a1a5422fb46fcc5e2eadbf4c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 37577 } } peers { permanent_uuid: "36021c39e44e48969a03206741895e19" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37155 } } peers { permanent_uuid: "83991525f5ad4c6c93cac4dc521832a8" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 33049 } }
I20260926 11:44:19.882993 25683 catalog_manager.cc:5495] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a9d9cf5a1a5422fb46fcc5e2eadbf4c (127.6.96.2). New cstate: current_term: 1 leader_uuid: "6a9d9cf5a1a5422fb46fcc5e2eadbf4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9d9cf5a1a5422fb46fcc5e2eadbf4c" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 37577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36021c39e44e48969a03206741895e19" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83991525f5ad4c6c93cac4dc521832a8" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 33049 } health_report { overall_health: UNKNOWN } } }
I20260926 11:44:19.924278 26156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:19.920620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46796 (local address 127.6.96.2:37577)
0926 11:44:19.920761 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:19.920764 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:19.920778 (+ 14us) server_negotiation.cc:407] Connection header received
0926 11:44:19.920852 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:19.920855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:19.920907 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:19.921014 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:44:19.921260 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.921777 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.922652 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:19.922975 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:19.923066 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:19.923385 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:44:19.923882 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:44:19.923984 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:19.924075 (+ 91us) server_negotiation.cc:299] Negotiation successful
0926 11:44:19.924132 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 11:44:19.930506 25829 raft_consensus.cc:1261] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 83991525f5ad4c6c93cac4dc521832a8 [term 1 FOLLOWER]: Refusing update from remote peer 6a9d9cf5a1a5422fb46fcc5e2eadbf4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:44:19.930510 26097 raft_consensus.cc:1261] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 36021c39e44e48969a03206741895e19 [term 1 FOLLOWER]: Refusing update from remote peer 6a9d9cf5a1a5422fb46fcc5e2eadbf4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:44:19.931282 26171 consensus_queue.cc:1035] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "83991525f5ad4c6c93cac4dc521832a8" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 33049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:44:19.931494 26168 consensus_queue.cc:1035] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "36021c39e44e48969a03206741895e19" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 37155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:44:19.941994 26175 mvcc.cc:205] Tried to move back new op lower bound from 7333572853467811840 to 7333572853268770816. Current Snapshot: MvccSnapshot[applied={T|T < 7333572853467811840}]
I20260926 11:44:19.948127 25683 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:44:19.948293 25683 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:44:19.950600 25683 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 78b48fb8aefd433a917191703b80c396: Sending DeleteTablet for 3 replicas of tablet 23d53c56f4fb4b13a3e2c5f2bdd0a94a
I20260926 11:44:19.951400 25943 tablet_service.cc:1514] Processing DeleteTablet for tablet 23d53c56f4fb4b13a3e2c5f2bdd0a94a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:44:19 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:46788
I20260926 11:44:19.951402 26077 tablet_service.cc:1514] Processing DeleteTablet for tablet 23d53c56f4fb4b13a3e2c5f2bdd0a94a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:44:19 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:41566
I20260926 11:44:19.951936 26183 tablet_replica.cc:330] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 36021c39e44e48969a03206741895e19: stopping tablet replica
I20260926 11:44:19.952126 26184 tablet_replica.cc:330] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c: stopping tablet replica
I20260926 11:44:19.952071 25809 tablet_service.cc:1514] Processing DeleteTablet for tablet 23d53c56f4fb4b13a3e2c5f2bdd0a94a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:44:19 UTC) from {username='kudu', principal='kudu/127.6.96.62@KRBTEST.COM'} at 127.0.0.1:38170
I20260926 11:44:19.952292 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 25746
I20260926 11:44:19.952610 26183 raft_consensus.cc:2229] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 36021c39e44e48969a03206741895e19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:44:19.952898 26184 raft_consensus.cc:2229] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c [term 1 LEADER]: Raft consensus shutting down.
I20260926 11:44:19.953325 26184 raft_consensus.cc:2258] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:44:19.953681 26183 raft_consensus.cc:2258] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 36021c39e44e48969a03206741895e19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:44:19.955440 26183 ts_tablet_manager.cc:1905] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 36021c39e44e48969a03206741895e19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:44:19.955444 26184 ts_tablet_manager.cc:1905] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:44:19.958073 26183 ts_tablet_manager.cc:1918] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 36021c39e44e48969a03206741895e19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:44:19.958084 26184 ts_tablet_manager.cc:1918] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:44:19.958165 26184 log.cc:1196] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-1/wal/wals/23d53c56f4fb4b13a3e2c5f2bdd0a94a
I20260926 11:44:19.958283 26183 log.cc:1196] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 36021c39e44e48969a03206741895e19: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/23d53c56f4fb4b13a3e2c5f2bdd0a94a
I20260926 11:44:19.958459 26184 ts_tablet_manager.cc:1939] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 6a9d9cf5a1a5422fb46fcc5e2eadbf4c: Deleting consensus metadata
I20260926 11:44:19.958716 26183 ts_tablet_manager.cc:1939] T 23d53c56f4fb4b13a3e2c5f2bdd0a94a P 36021c39e44e48969a03206741895e19: Deleting consensus metadata
I20260926 11:44:19.959822 25668 catalog_manager.cc:4841] TS 6a9d9cf5a1a5422fb46fcc5e2eadbf4c (127.6.96.2:37577): tablet 23d53c56f4fb4b13a3e2c5f2bdd0a94a (table test-table [id=78f88a93848243eab5d4717e0dfeed41]) successfully deleted
I20260926 11:44:19.959868 25669 catalog_manager.cc:4841] TS 36021c39e44e48969a03206741895e19 (127.6.96.3:37155): tablet 23d53c56f4fb4b13a3e2c5f2bdd0a94a (table test-table [id=78f88a93848243eab5d4717e0dfeed41]) successfully deleted
W20260926 11:44:19.960441 25671 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.96.1:33049 (error 108)
I20260926 11:44:19.961205 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 25880
I20260926 11:44:19.961445 26157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:19.961173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:33049 (local address 127.0.0.1:38176)
0926 11:44:19.961288 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:19.961359 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.1:33049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260926 11:44:19.961786 25671 catalog_manager.cc:4584] TS 83991525f5ad4c6c93cac4dc521832a8 (127.6.96.1:33049): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 23d53c56f4fb4b13a3e2c5f2bdd0a94a: Network error: Client connection negotiation failed: client connection to 127.6.96.1:33049: connect: Connection refused (error 111)
I20260926 11:44:19.966408 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 26014
I20260926 11:44:19.972050 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 25652
2026-09-26T11:44:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2447 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-26T11:44:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:19Z Disabled control of system clock
I20260926 11:44:20.000005 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:36907
--webserver_interface=127.6.96.62
--webserver_port=0
--builtin_ntp_servers=127.6.96.20:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:36907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 11:44:20.108613 26189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:20.108896 26189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:20.108932 26189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:20.113206 26189 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:20.113276 26189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:20.113289 26189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:20.113306 26189 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:20.113320 26189 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:20.118218 26189 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:36907
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:36907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:20.119318 26189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:20.120675 26189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 11:44:20.129565 26189 server_base.cc:1034] running on GCE node
W20260926 11:44:20.129572 26194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:20.129585 26195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:20.129808 26197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:20.130388 26189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:20.131578 26189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:20.132761 26189 hybrid_clock.cc:648] HybridClock initialized: now 1790423060132730 us; error 45 us; skew 500 ppm
I20260926 11:44:20.134725 26189 webserver.cc:460] Webserver started at http://127.6.96.62:45789/ using document root <none> and password file <none>
I20260926 11:44:20.135344 26189 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:20.135414 26189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:20.135610 26189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:20.137694 26189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "b41aaffa8eed4b8c98532c2cc3377ffa"
format_stamp: "Formatted at 2026-09-26 11:44:20 on dist-test-slave-xxxx"
server_key: "f3c2ffbdbacedc18928235f0e28d626d"
server_key_iv: "2bb99675b0f13ef1fa4397baa80b41ce"
server_key_version: "encryptionkey@0"
I20260926 11:44:20.138250 26189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "b41aaffa8eed4b8c98532c2cc3377ffa"
format_stamp: "Formatted at 2026-09-26 11:44:20 on dist-test-slave-xxxx"
server_key: "f3c2ffbdbacedc18928235f0e28d626d"
server_key_iv: "2bb99675b0f13ef1fa4397baa80b41ce"
server_key_version: "encryptionkey@0"
I20260926 11:44:20.141981 26189 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 11:44:20.144574 26203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:20.145895 26189 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 11:44:20.146016 26189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "b41aaffa8eed4b8c98532c2cc3377ffa"
format_stamp: "Formatted at 2026-09-26 11:44:20 on dist-test-slave-xxxx"
server_key: "f3c2ffbdbacedc18928235f0e28d626d"
server_key_iv: "2bb99675b0f13ef1fa4397baa80b41ce"
server_key_version: "encryptionkey@0"
I20260926 11:44:20.146123 26189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:20.165748 26189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:20.166532 26189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:20.166740 26189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:20.173935 26189 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:36907
I20260926 11:44:20.173947 26255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:36907 every 8 connection(s)
I20260926 11:44:20.174952 26189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:20.175634 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 26189
I20260926 11:44:20.175765 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:20.176115 6528 external_mini_cluster.cc:1428] Setting key d9e8d59790e4f632b8a81fdac8a74847
I20260926 11:44:20.178615 26256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:20.184365 26256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b41aaffa8eed4b8c98532c2cc3377ffa: Bootstrap starting.
I20260926 11:44:20.185302 26259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:20.177586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51592 (local address 127.6.96.62:36907)
0926 11:44:20.178236 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:20.178251 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:20.178288 (+ 37us) server_negotiation.cc:407] Connection header received
0926 11:44:20.179024 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:20.179044 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:20.179470 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:20.179789 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:20.180219 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.181188 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.182074 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.182549 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.182692 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:20.183576 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:20.183581 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:20.183593 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:20.183983 (+ 390us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:20.184039 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:20.184058 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:20.184141 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:20.184279 (+ 138us) server_negotiation.cc:299] Negotiation successful
0926 11:44:20.184571 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":147,"threads_started":1}
I20260926 11:44:20.186677 26256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b41aaffa8eed4b8c98532c2cc3377ffa: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:20.187513 26256 log.cc:824] T 00000000000000000000000000000000 P b41aaffa8eed4b8c98532c2cc3377ffa: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:20.189395 26256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b41aaffa8eed4b8c98532c2cc3377ffa: No bootstrap required, opened a new log
I20260926 11:44:20.192981 26256 raft_consensus.cc:348] T 00000000000000000000000000000000 P b41aaffa8eed4b8c98532c2cc3377ffa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b41aaffa8eed4b8c98532c2cc3377ffa" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36907 } }
I20260926 11:44:20.193249 26256 raft_consensus.cc:374] T 00000000000000000000000000000000 P b41aaffa8eed4b8c98532c2cc3377ffa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:20.193308 26256 raft_consensus.cc:729] T 00000000000000000000000000000000 P b41aaffa8eed4b8c98532c2cc3377ffa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b41aaffa8eed4b8c98532c2cc3377ffa, State: Initialized, Role: FOLLOWER
I20260926 11:44:20.193749 26256 consensus_queue.cc:260] T 00000000000000000000000000000000 P b41aaffa8eed4b8c98532c2cc3377ffa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b41aaffa8eed4b8c98532c2cc3377ffa" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36907 } }
I20260926 11:44:20.193881 26256 raft_consensus.cc:388] T 00000000000000000000000000000000 P b41aaffa8eed4b8c98532c2cc3377ffa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:44:20.193945 26256 raft_consensus.cc:482] T 00000000000000000000000000000000 P b41aaffa8eed4b8c98532c2cc3377ffa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:44:20.194051 26256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b41aaffa8eed4b8c98532c2cc3377ffa [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:20.195006 26256 raft_consensus.cc:504] T 00000000000000000000000000000000 P b41aaffa8eed4b8c98532c2cc3377ffa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b41aaffa8eed4b8c98532c2cc3377ffa" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36907 } }
I20260926 11:44:20.195336 26256 leader_election.cc:302] T 00000000000000000000000000000000 P b41aaffa8eed4b8c98532c2cc3377ffa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b41aaffa8eed4b8c98532c2cc3377ffa; no voters:
I20260926 11:44:20.195634 26256 leader_election.cc:288] T 00000000000000000000000000000000 P b41aaffa8eed4b8c98532c2cc3377ffa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:44:20.195781 26261 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b41aaffa8eed4b8c98532c2cc3377ffa [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:20.196007 26261 raft_consensus.cc:686] T 00000000000000000000000000000000 P b41aaffa8eed4b8c98532c2cc3377ffa [term 1 LEADER]: Becoming Leader. State: Replica: b41aaffa8eed4b8c98532c2cc3377ffa, State: Running, Role: LEADER
I20260926 11:44:20.196295 26261 consensus_queue.cc:237] T 00000000000000000000000000000000 P b41aaffa8eed4b8c98532c2cc3377ffa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b41aaffa8eed4b8c98532c2cc3377ffa" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36907 } }
I20260926 11:44:20.196811 26256 sys_catalog.cc:564] T 00000000000000000000000000000000 P b41aaffa8eed4b8c98532c2cc3377ffa [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:20.198097 26263 sys_catalog.cc:455] T 00000000000000000000000000000000 P b41aaffa8eed4b8c98532c2cc3377ffa [sys.catalog]: SysCatalogTable state changed. Reason: New leader b41aaffa8eed4b8c98532c2cc3377ffa. Latest consensus state: current_term: 1 leader_uuid: "b41aaffa8eed4b8c98532c2cc3377ffa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b41aaffa8eed4b8c98532c2cc3377ffa" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36907 } } }
I20260926 11:44:20.198112 26262 sys_catalog.cc:455] T 00000000000000000000000000000000 P b41aaffa8eed4b8c98532c2cc3377ffa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b41aaffa8eed4b8c98532c2cc3377ffa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b41aaffa8eed4b8c98532c2cc3377ffa" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 36907 } } }
I20260926 11:44:20.198240 26263 sys_catalog.cc:458] T 00000000000000000000000000000000 P b41aaffa8eed4b8c98532c2cc3377ffa [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:20.198259 26262 sys_catalog.cc:458] T 00000000000000000000000000000000 P b41aaffa8eed4b8c98532c2cc3377ffa [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:20.198599 26271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:20.201742 26271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:20.207600 26271 catalog_manager.cc:1347] Generated new cluster ID: 3269337587ae47febddb66979ddaf808
I20260926 11:44:20.207671 26271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:20.214794 26271 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:20.215793 26271 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:20.222121 26271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b41aaffa8eed4b8c98532c2cc3377ffa: Generated new TSK 0
I20260926 11:44:20.222781 26271 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:44:20.227322 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.1:0
--local_ip_for_outbound_sockets=127.6.96.1
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:36907
--builtin_ntp_servers=127.6.96.20:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 11:44:20.339859 26280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:20.340186 26280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:20.340227 26280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:20.344740 26280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:20.344827 26280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:20.344888 26280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.1
I20260926 11:44:20.350342 26280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.96.1
--webserver_port=0
--tserver_master_addrs=127.6.96.62:36907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.1
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:20.351643 26280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:20.353560 26280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 11:44:20.363864 26280 server_base.cc:1034] running on GCE node
W20260926 11:44:20.363806 26288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:20.363806 26286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:20.363765 26285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:20.364507 26280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:20.365098 26280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:20.366268 26280 hybrid_clock.cc:648] HybridClock initialized: now 1790423060366227 us; error 41 us; skew 500 ppm
I20260926 11:44:20.367992 26280 webserver.cc:460] Webserver started at http://127.6.96.1:42513/ using document root <none> and password file <none>
I20260926 11:44:20.368530 26280 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:20.368587 26280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:20.368779 26280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:20.370755 26280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-0/data/instance:
uuid: "7080253319db497cbf4e292cbe5245dd"
format_stamp: "Formatted at 2026-09-26 11:44:20 on dist-test-slave-xxxx"
server_key: "b45a5a1789998374c2e241e4ad11e9e3"
server_key_iv: "0d88fb2ac12aa32b2c46e7fd9bcc20ed"
server_key_version: "encryptionkey@0"
I20260926 11:44:20.371318 26280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance:
uuid: "7080253319db497cbf4e292cbe5245dd"
format_stamp: "Formatted at 2026-09-26 11:44:20 on dist-test-slave-xxxx"
server_key: "b45a5a1789998374c2e241e4ad11e9e3"
server_key_iv: "0d88fb2ac12aa32b2c46e7fd9bcc20ed"
server_key_version: "encryptionkey@0"
I20260926 11:44:20.374796 26280 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:44:20.377065 26294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:20.378213 26280 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:44:20.378319 26280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-0/wal
uuid: "7080253319db497cbf4e292cbe5245dd"
format_stamp: "Formatted at 2026-09-26 11:44:20 on dist-test-slave-xxxx"
server_key: "b45a5a1789998374c2e241e4ad11e9e3"
server_key_iv: "0d88fb2ac12aa32b2c46e7fd9bcc20ed"
server_key_version: "encryptionkey@0"
I20260926 11:44:20.378407 26280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:20.416029 26280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:20.416759 26280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:20.417172 26280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:20.417848 26280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:44:20.418792 26280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:44:20.418859 26280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:20.418920 26280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:44:20.418949 26280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:20.427284 26280 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.1:43929
I20260926 11:44:20.427306 26407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.1:43929 every 8 connection(s)
I20260926 11:44:20.428184 26280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-0/data/info.pb
I20260926 11:44:20.433732 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 26280
I20260926 11:44:20.433871 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-0/wal/instance
I20260926 11:44:20.434139 6528 external_mini_cluster.cc:1428] Setting key 9e70703da3b3a95ee8c86bce873bc3c9
I20260926 11:44:20.436172 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.2:0
--local_ip_for_outbound_sockets=127.6.96.2
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:36907
--builtin_ntp_servers=127.6.96.20:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 11:44:20.437680 26259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:20.429925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.1:43505 (local address 127.6.96.62:36907)
0926 11:44:20.430084 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:20.430088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:20.431220 (+ 1132us) server_negotiation.cc:407] Connection header received
0926 11:44:20.432562 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:20.432566 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:20.432618 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:20.432708 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:20.433658 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.434276 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.435662 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.435875 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.435973 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:20.437153 (+ 1180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:20.437156 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:20.437158 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:20.437170 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:20.437190 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:20.437198 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:20.437249 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:20.437435 (+ 186us) server_negotiation.cc:299] Negotiation successful
0926 11:44:20.437528 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 11:44:20.438531 26410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:20.430349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:36907 (local address 127.6.96.1:43505)
0926 11:44:20.431072 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:20.431115 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:20.432352 (+ 1237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:20.432862 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:20.432868 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:20.432981 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:20.433502 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:20.433521 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.434412 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.434416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:20.435555 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:20.435566 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.436029 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.436033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:20.436248 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:20.436840 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 11:44:20.436842 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:20.436907 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 11:44:20.436994 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:20.437265 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:20.437272 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:20.437277 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 11:44:20.437412 (+ 135us) client_negotiation.cc:240] Negotiation successful
0926 11:44:20.437695 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":104,"threads_started":1}
I20260926 11:44:20.439949 26408 heartbeater.cc:347] Connected to a master server at 127.6.96.62:36907
I20260926 11:44:20.440193 26408 heartbeater.cc:464] Registering TS with master...
I20260926 11:44:20.440702 26408 heartbeater.cc:511] Master 127.6.96.62:36907 requested a full tablet report, sending...
I20260926 11:44:20.442250 26220 ts_manager.cc:194] Registered new tserver with Master: 7080253319db497cbf4e292cbe5245dd (127.6.96.1:43929)
I20260926 11:44:20.443485 26220 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.96.1:43505
W20260926 11:44:20.546330 26411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:20.546598 26411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:20.546636 26411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:20.550879 26411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:20.550943 26411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:20.551008 26411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.2
I20260926 11:44:20.555924 26411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.96.2
--webserver_port=0
--tserver_master_addrs=127.6.96.62:36907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.2
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:20.557026 26411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:20.558399 26411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 11:44:20.567811 26411 server_base.cc:1034] running on GCE node
W20260926 11:44:20.567762 26416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:20.567762 26417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:20.567766 26419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:20.568428 26411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:20.569084 26411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:20.570233 26411 hybrid_clock.cc:648] HybridClock initialized: now 1790423060570211 us; error 35 us; skew 500 ppm
I20260926 11:44:20.572074 26411 webserver.cc:460] Webserver started at http://127.6.96.2:43099/ using document root <none> and password file <none>
I20260926 11:44:20.572629 26411 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:20.572695 26411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:20.572872 26411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:20.574940 26411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-1/data/instance:
uuid: "20152320df954b4c889d1d6b505a953b"
format_stamp: "Formatted at 2026-09-26 11:44:20 on dist-test-slave-xxxx"
server_key: "945e3aa4f0b167f3f03b320f29ef25ec"
server_key_iv: "5275db5d3bb4903e11e4c77003391a50"
server_key_version: "encryptionkey@0"
I20260926 11:44:20.575475 26411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance:
uuid: "20152320df954b4c889d1d6b505a953b"
format_stamp: "Formatted at 2026-09-26 11:44:20 on dist-test-slave-xxxx"
server_key: "945e3aa4f0b167f3f03b320f29ef25ec"
server_key_iv: "5275db5d3bb4903e11e4c77003391a50"
server_key_version: "encryptionkey@0"
I20260926 11:44:20.579016 26411 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 11:44:20.581447 26425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:20.582744 26411 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:44:20.582878 26411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-1/wal
uuid: "20152320df954b4c889d1d6b505a953b"
format_stamp: "Formatted at 2026-09-26 11:44:20 on dist-test-slave-xxxx"
server_key: "945e3aa4f0b167f3f03b320f29ef25ec"
server_key_iv: "5275db5d3bb4903e11e4c77003391a50"
server_key_version: "encryptionkey@0"
I20260926 11:44:20.582993 26411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:20.616310 26411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:20.617038 26411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:20.617210 26411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:20.617817 26411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:44:20.618773 26411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:44:20.618849 26411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:20.618906 26411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:44:20.618938 26411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:20.627249 26411 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.2:43335
I20260926 11:44:20.627292 26538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.2:43335 every 8 connection(s)
I20260926 11:44:20.628160 26411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-1/data/info.pb
I20260926 11:44:20.632860 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 26411
I20260926 11:44:20.632994 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-1/wal/instance
I20260926 11:44:20.633260 6528 external_mini_cluster.cc:1428] Setting key be74108eda9b4dd9da11182503c50fc6
I20260926 11:44:20.635221 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.96.3:0
--local_ip_for_outbound_sockets=127.6.96.3
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:36907
--builtin_ntp_servers=127.6.96.20:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 11:44:20.637364 26259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:20.629917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.2:56125 (local address 127.6.96.62:36907)
0926 11:44:20.630054 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:20.630057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:20.631214 (+ 1157us) server_negotiation.cc:407] Connection header received
0926 11:44:20.632529 (+ 1315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:20.632532 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:20.632585 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:20.632678 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:20.633623 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.634208 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.635503 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.635808 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.635909 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:20.636918 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:20.636921 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:20.636924 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:20.636937 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:20.636956 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:20.636964 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:20.637020 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:20.637154 (+ 134us) server_negotiation.cc:299] Negotiation successful
0926 11:44:20.637214 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 11:44:20.638212 26541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:20.630322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:36907 (local address 127.6.96.2:56125)
0926 11:44:20.631086 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:20.631127 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:20.632346 (+ 1219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:20.632857 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:20.632863 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:20.632952 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:20.633487 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:20.633501 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.634327 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.634330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:20.635384 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:20.635396 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.635941 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.635943 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:20.636086 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:20.636646 (+ 560us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 11:44:20.636647 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:20.636688 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 11:44:20.636752 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:20.637031 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:20.637040 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:20.637046 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 11:44:20.637149 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 11:44:20.637434 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":113,"threads_started":1}
I20260926 11:44:20.639376 26539 heartbeater.cc:347] Connected to a master server at 127.6.96.62:36907
I20260926 11:44:20.639590 26539 heartbeater.cc:464] Registering TS with master...
I20260926 11:44:20.640110 26539 heartbeater.cc:511] Master 127.6.96.62:36907 requested a full tablet report, sending...
I20260926 11:44:20.641194 26220 ts_manager.cc:194] Registered new tserver with Master: 20152320df954b4c889d1d6b505a953b (127.6.96.2:43335)
I20260926 11:44:20.641956 26220 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.96.2:56125
W20260926 11:44:20.747187 26542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:20.747431 26542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:20.747470 26542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:20.751546 26542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:20.751595 26542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:20.751647 26542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.96.3
I20260926 11:44:20.756245 26542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.96.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.96.3
--webserver_port=0
--tserver_master_addrs=127.6.96.62:36907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.96.3
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:20.757217 26542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:20.758512 26542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:20.767686 26548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:20.767694 26547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:20.767848 26542 server_base.cc:1034] running on GCE node
W20260926 11:44:20.767732 26550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:20.768402 26542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:20.768989 26542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:20.770138 26542 hybrid_clock.cc:648] HybridClock initialized: now 1790423060770117 us; error 36 us; skew 500 ppm
I20260926 11:44:20.771855 26542 webserver.cc:460] Webserver started at http://127.6.96.3:40405/ using document root <none> and password file <none>
I20260926 11:44:20.772383 26542 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:20.772456 26542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:20.772648 26542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:20.774580 26542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-2/data/instance:
uuid: "1d9c79a759a24889a719fdd27098fdaf"
format_stamp: "Formatted at 2026-09-26 11:44:20 on dist-test-slave-xxxx"
server_key: "50bf601b1089c99cdfbc3cda40755258"
server_key_iv: "ac42ccda536be6a57fb83cb53e083acb"
server_key_version: "encryptionkey@0"
I20260926 11:44:20.775203 26542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance:
uuid: "1d9c79a759a24889a719fdd27098fdaf"
format_stamp: "Formatted at 2026-09-26 11:44:20 on dist-test-slave-xxxx"
server_key: "50bf601b1089c99cdfbc3cda40755258"
server_key_iv: "ac42ccda536be6a57fb83cb53e083acb"
server_key_version: "encryptionkey@0"
I20260926 11:44:20.778564 26542 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:44:20.780833 26556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:20.782049 26542 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:44:20.782140 26542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-2/wal
uuid: "1d9c79a759a24889a719fdd27098fdaf"
format_stamp: "Formatted at 2026-09-26 11:44:20 on dist-test-slave-xxxx"
server_key: "50bf601b1089c99cdfbc3cda40755258"
server_key_iv: "ac42ccda536be6a57fb83cb53e083acb"
server_key_version: "encryptionkey@0"
I20260926 11:44:20.782223 26542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:20.795912 26542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:20.796586 26542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:20.796736 26542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:20.797312 26542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 11:44:20.798285 26542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 11:44:20.798331 26542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:20.798370 26542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 11:44:20.798389 26542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:20.807236 26542 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.3:36997
I20260926 11:44:20.807258 26669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.3:36997 every 8 connection(s)
I20260926 11:44:20.808162 26542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-2/data/info.pb
I20260926 11:44:20.811275 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 26542
I20260926 11:44:20.811412 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-2/wal/instance
I20260926 11:44:20.811734 6528 external_mini_cluster.cc:1428] Setting key 7a954a313aa3e3b6f59616f06a5f7872
I20260926 11:44:20.817813 26259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:20.810091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:57787 (local address 127.6.96.62:36907)
0926 11:44:20.810250 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:20.810254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:20.811453 (+ 1199us) server_negotiation.cc:407] Connection header received
0926 11:44:20.813098 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:20.813101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:20.813153 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:20.813234 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:20.814215 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.814750 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.816052 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.816277 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.816362 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:20.817414 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:20.817417 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:20.817419 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:20.817430 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:20.817450 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:20.817457 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:20.817507 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:20.817648 (+ 141us) server_negotiation.cc:299] Negotiation successful
0926 11:44:20.817697 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 11:44:20.818815 26672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:20.810480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:36907 (local address 127.6.96.3:57787)
0926 11:44:20.811274 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:20.811330 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:20.812882 (+ 1552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:20.813385 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:20.813391 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:20.813481 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:20.814088 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:20.814103 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.814907 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.814911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:20.815917 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:20.815927 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.816419 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.816422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:20.816568 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:20.817184 (+ 616us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 11:44:20.817185 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:20.817230 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 11:44:20.817294 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:20.817528 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:20.817540 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:20.817546 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 11:44:20.817641 (+ 95us) client_negotiation.cc:240] Negotiation successful
0926 11:44:20.817899 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":110,"threads_started":1}
I20260926 11:44:20.819945 26670 heartbeater.cc:347] Connected to a master server at 127.6.96.62:36907
I20260926 11:44:20.820185 26670 heartbeater.cc:464] Registering TS with master...
I20260926 11:44:20.820706 26670 heartbeater.cc:511] Master 127.6.96.62:36907 requested a full tablet report, sending...
I20260926 11:44:20.821688 26220 ts_manager.cc:194] Registered new tserver with Master: 1d9c79a759a24889a719fdd27098fdaf (127.6.96.3:36997)
I20260926 11:44:20.822217 26220 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.96.3:57787
I20260926 11:44:20.825464 6528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 11:44:20.830585 26259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:20.827325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51596 (local address 127.6.96.62:36907)
0926 11:44:20.827459 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:20.827463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:20.827624 (+ 161us) server_negotiation.cc:407] Connection header received
0926 11:44:20.827733 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:20.827736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:20.827778 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:20.827839 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:20.828034 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.828527 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.829289 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.829500 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.829569 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:20.830238 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:20.830241 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:20.830243 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:20.830254 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:20.830273 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:20.830281 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:20.830332 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:20.830390 (+ 58us) server_negotiation.cc:299] Negotiation successful
0926 11:44:20.830447 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 11:44:20.833293 26220 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51596:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 11:44:20.836263 26220 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 11:44:20.849782 26681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:20.844102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:43335 (local address 127.0.0.1:48864)
0926 11:44:20.844987 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:20.845036 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:20.845151 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:20.845716 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:20.845720 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:20.845738 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:20.845844 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:20.845849 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.847031 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.847034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:20.848187 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:20.848197 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.849270 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.849274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:20.849374 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:20.849409 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 11:44:20.849518 (+ 109us) client_negotiation.cc:240] Negotiation successful
0926 11:44:20.849634 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"thread_start_us":57,"threads_started":1}
I20260926 11:44:20.849843 26683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:20.844285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48864 (local address 127.6.96.2:43335)
0926 11:44:20.845233 (+ 948us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:20.845244 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:20.845269 (+ 25us) server_negotiation.cc:407] Connection header received
0926 11:44:20.845347 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:20.845352 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:20.845563 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:20.845722 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:20.845975 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.846931 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.848315 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.849167 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.849257 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:20.849449 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:20.849548 (+ 99us) server_negotiation.cc:299] Negotiation successful
0926 11:44:20.849642 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":833,"thread_start_us":110,"threads_started":1}
I20260926 11:44:20.850441 26686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:20.844869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43786 (local address 127.6.96.1:43929)
0926 11:44:20.845393 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:20.845402 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:20.845520 (+ 118us) server_negotiation.cc:407] Connection header received
0926 11:44:20.845593 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:20.845598 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:20.845786 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:20.845928 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:20.846167 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.847305 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.848706 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.849318 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.849415 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:20.849589 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:20.850185 (+ 596us) server_negotiation.cc:299] Negotiation successful
0926 11:44:20.850285 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":101,"threads_started":1}
I20260926 11:44:20.850088 26682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:20.844185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:43929 (local address 127.0.0.1:43786)
0926 11:44:20.845427 (+ 1242us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:20.845438 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:20.845517 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:20.845914 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:20.845916 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:20.845949 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:20.846042 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:20.846048 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.847574 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.847577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:20.848603 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:20.848611 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.849650 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.849654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:20.849785 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:20.849809 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 11:44:20.849878 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 11:44:20.849956 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1182,"mutex_wait_us":1,"thread_start_us":97,"threads_started":1}
I20260926 11:44:20.850579 26684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:20.844432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.3:36997 (local address 127.0.0.1:38376)
0926 11:44:20.845948 (+ 1516us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:20.845960 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:20.846043 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:20.846526 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:20.846529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:20.846547 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:20.846633 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:20.846638 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.847905 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.847908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:20.848997 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:20.849008 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.850223 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.850227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:20.850306 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:20.850328 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 11:44:20.850396 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 11:44:20.850471 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1455,"thread_start_us":81,"threads_started":1}
I20260926 11:44:20.851291 26685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:20.844779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38376 (local address 127.6.96.3:36997)
0926 11:44:20.845222 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:20.845234 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:20.846132 (+ 898us) server_negotiation.cc:407] Connection header received
0926 11:44:20.846205 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:20.846221 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:20.846394 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:20.846535 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:20.846745 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.847794 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.849149 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.849890 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.849977 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:20.850136 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:20.851053 (+ 917us) server_negotiation.cc:299] Negotiation successful
0926 11:44:20.851145 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":131,"threads_started":1}
I20260926 11:44:20.853086 26342 tablet_service.cc:1467] Processing CreateTablet for tablet 74417ef830f94e1186f6c39d10f0e039 (DEFAULT_TABLE table=test-table [id=6647b26cb33a41a6866b5b2b20300225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:44:20.853117 26473 tablet_service.cc:1467] Processing CreateTablet for tablet 74417ef830f94e1186f6c39d10f0e039 (DEFAULT_TABLE table=test-table [id=6647b26cb33a41a6866b5b2b20300225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:44:20.854064 26342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74417ef830f94e1186f6c39d10f0e039. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:20.853689 26604 tablet_service.cc:1467] Processing CreateTablet for tablet 74417ef830f94e1186f6c39d10f0e039 (DEFAULT_TABLE table=test-table [id=6647b26cb33a41a6866b5b2b20300225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 11:44:20.854115 26473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74417ef830f94e1186f6c39d10f0e039. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:20.854555 26604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74417ef830f94e1186f6c39d10f0e039. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:20.860679 26687 tablet_bootstrap.cc:492] T 74417ef830f94e1186f6c39d10f0e039 P 7080253319db497cbf4e292cbe5245dd: Bootstrap starting.
I20260926 11:44:20.861423 26688 tablet_bootstrap.cc:492] T 74417ef830f94e1186f6c39d10f0e039 P 20152320df954b4c889d1d6b505a953b: Bootstrap starting.
I20260926 11:44:20.862668 26687 tablet_bootstrap.cc:654] T 74417ef830f94e1186f6c39d10f0e039 P 7080253319db497cbf4e292cbe5245dd: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:20.863103 26688 tablet_bootstrap.cc:654] T 74417ef830f94e1186f6c39d10f0e039 P 20152320df954b4c889d1d6b505a953b: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:20.863418 26687 log.cc:824] T 74417ef830f94e1186f6c39d10f0e039 P 7080253319db497cbf4e292cbe5245dd: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:20.863744 26688 log.cc:824] T 74417ef830f94e1186f6c39d10f0e039 P 20152320df954b4c889d1d6b505a953b: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:20.864019 26689 tablet_bootstrap.cc:492] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf: Bootstrap starting.
I20260926 11:44:20.865283 26687 tablet_bootstrap.cc:492] T 74417ef830f94e1186f6c39d10f0e039 P 7080253319db497cbf4e292cbe5245dd: No bootstrap required, opened a new log
I20260926 11:44:20.865478 26687 ts_tablet_manager.cc:1397] T 74417ef830f94e1186f6c39d10f0e039 P 7080253319db497cbf4e292cbe5245dd: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260926 11:44:20.865656 26688 tablet_bootstrap.cc:492] T 74417ef830f94e1186f6c39d10f0e039 P 20152320df954b4c889d1d6b505a953b: No bootstrap required, opened a new log
I20260926 11:44:20.865832 26688 ts_tablet_manager.cc:1397] T 74417ef830f94e1186f6c39d10f0e039 P 20152320df954b4c889d1d6b505a953b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 11:44:20.865885 26689 tablet_bootstrap.cc:654] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:20.866577 26689 log.cc:824] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:20.868113 26689 tablet_bootstrap.cc:492] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf: No bootstrap required, opened a new log
I20260926 11:44:20.868281 26689 ts_tablet_manager.cc:1397] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260926 11:44:20.869724 26687 raft_consensus.cc:348] T 74417ef830f94e1186f6c39d10f0e039 P 7080253319db497cbf4e292cbe5245dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20152320df954b4c889d1d6b505a953b" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 43335 } } peers { permanent_uuid: "7080253319db497cbf4e292cbe5245dd" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 43929 } } peers { permanent_uuid: "1d9c79a759a24889a719fdd27098fdaf" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36997 } }
I20260926 11:44:20.869729 26688 raft_consensus.cc:348] T 74417ef830f94e1186f6c39d10f0e039 P 20152320df954b4c889d1d6b505a953b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20152320df954b4c889d1d6b505a953b" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 43335 } } peers { permanent_uuid: "7080253319db497cbf4e292cbe5245dd" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 43929 } } peers { permanent_uuid: "1d9c79a759a24889a719fdd27098fdaf" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36997 } }
I20260926 11:44:20.869985 26687 raft_consensus.cc:374] T 74417ef830f94e1186f6c39d10f0e039 P 7080253319db497cbf4e292cbe5245dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:20.869985 26688 raft_consensus.cc:374] T 74417ef830f94e1186f6c39d10f0e039 P 20152320df954b4c889d1d6b505a953b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:20.870039 26687 raft_consensus.cc:729] T 74417ef830f94e1186f6c39d10f0e039 P 7080253319db497cbf4e292cbe5245dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7080253319db497cbf4e292cbe5245dd, State: Initialized, Role: FOLLOWER
I20260926 11:44:20.870039 26688 raft_consensus.cc:729] T 74417ef830f94e1186f6c39d10f0e039 P 20152320df954b4c889d1d6b505a953b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20152320df954b4c889d1d6b505a953b, State: Initialized, Role: FOLLOWER
I20260926 11:44:20.870489 26688 consensus_queue.cc:260] T 74417ef830f94e1186f6c39d10f0e039 P 20152320df954b4c889d1d6b505a953b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20152320df954b4c889d1d6b505a953b" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 43335 } } peers { permanent_uuid: "7080253319db497cbf4e292cbe5245dd" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 43929 } } peers { permanent_uuid: "1d9c79a759a24889a719fdd27098fdaf" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36997 } }
I20260926 11:44:20.870489 26687 consensus_queue.cc:260] T 74417ef830f94e1186f6c39d10f0e039 P 7080253319db497cbf4e292cbe5245dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20152320df954b4c889d1d6b505a953b" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 43335 } } peers { permanent_uuid: "7080253319db497cbf4e292cbe5245dd" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 43929 } } peers { permanent_uuid: "1d9c79a759a24889a719fdd27098fdaf" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36997 } }
I20260926 11:44:20.871497 26687 ts_tablet_manager.cc:1428] T 74417ef830f94e1186f6c39d10f0e039 P 7080253319db497cbf4e292cbe5245dd: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260926 11:44:20.871497 26688 ts_tablet_manager.cc:1428] T 74417ef830f94e1186f6c39d10f0e039 P 20152320df954b4c889d1d6b505a953b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 11:44:20.871510 26539 heartbeater.cc:503] Master 127.6.96.62:36907 was elected leader, sending a full tablet report...
I20260926 11:44:20.871743 26689 raft_consensus.cc:348] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20152320df954b4c889d1d6b505a953b" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 43335 } } peers { permanent_uuid: "7080253319db497cbf4e292cbe5245dd" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 43929 } } peers { permanent_uuid: "1d9c79a759a24889a719fdd27098fdaf" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36997 } }
I20260926 11:44:20.871982 26689 raft_consensus.cc:374] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:20.872045 26689 raft_consensus.cc:729] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d9c79a759a24889a719fdd27098fdaf, State: Initialized, Role: FOLLOWER
I20260926 11:44:20.872133 26408 heartbeater.cc:503] Master 127.6.96.62:36907 was elected leader, sending a full tablet report...
I20260926 11:44:20.872450 26689 consensus_queue.cc:260] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20152320df954b4c889d1d6b505a953b" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 43335 } } peers { permanent_uuid: "7080253319db497cbf4e292cbe5245dd" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 43929 } } peers { permanent_uuid: "1d9c79a759a24889a719fdd27098fdaf" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36997 } }
I20260926 11:44:20.873353 26689 ts_tablet_manager.cc:1428] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260926 11:44:20.873684 26670 heartbeater.cc:503] Master 127.6.96.62:36907 was elected leader, sending a full tablet report...
W20260926 11:44:20.878983 26540 tablet.cc:2369] T 74417ef830f94e1186f6c39d10f0e039 P 20152320df954b4c889d1d6b505a953b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 11:44:20.929481 26409 tablet.cc:2369] T 74417ef830f94e1186f6c39d10f0e039 P 7080253319db497cbf4e292cbe5245dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:44:20.949009 26695 raft_consensus.cc:482] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 11:44:20.949231 26695 raft_consensus.cc:504] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20152320df954b4c889d1d6b505a953b" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 43335 } } peers { permanent_uuid: "7080253319db497cbf4e292cbe5245dd" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 43929 } } peers { permanent_uuid: "1d9c79a759a24889a719fdd27098fdaf" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36997 } }
I20260926 11:44:20.950429 26695 leader_election.cc:288] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20152320df954b4c889d1d6b505a953b (127.6.96.2:43335), 7080253319db497cbf4e292cbe5245dd (127.6.96.1:43929)
I20260926 11:44:20.953889 26686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:20.950785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:39851 (local address 127.6.96.1:43929)
0926 11:44:20.950926 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:20.950930 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:20.950998 (+ 68us) server_negotiation.cc:407] Connection header received
0926 11:44:20.951132 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:20.951134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:20.951184 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:20.951246 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:20.951494 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.951992 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.953037 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.953412 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.953502 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:20.953561 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:20.953721 (+ 160us) server_negotiation.cc:299] Negotiation successful
0926 11:44:20.953782 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 11:44:20.953889 26672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:20.950693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:43929 (local address 127.6.96.3:39851)
0926 11:44:20.950909 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:20.950925 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:20.951027 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:20.951269 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:20.951271 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:20.951287 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:20.951393 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:20.951398 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.952094 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.952097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:20.952924 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:20.952933 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.953516 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.953520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:20.953611 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:20.953629 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 11:44:20.953691 (+ 62us) client_negotiation.cc:240] Negotiation successful
0926 11:44:20.953758 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":42}
I20260926 11:44:20.953964 26683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:20.950802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.96.3:34425 (local address 127.6.96.2:43335)
0926 11:44:20.950954 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:20.950958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:20.951214 (+ 256us) server_negotiation.cc:407] Connection header received
0926 11:44:20.951308 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:20.951311 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:20.951353 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:20.951412 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 11:44:20.951645 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.952123 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.953095 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.953469 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.953555 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:20.953609 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:20.953789 (+ 180us) server_negotiation.cc:299] Negotiation successful
0926 11:44:20.953854 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260926 11:44:20.954001 26696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:20.950692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:43335 (local address 127.6.96.3:34425)
0926 11:44:20.951114 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:20.951124 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:20.951226 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:20.951428 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:20.951430 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:20.951445 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 11:44:20.951550 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:20.951555 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.952220 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.952224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:20.952985 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:20.952993 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.953580 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.953583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:20.953675 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:20.953691 (+ 16us) client_negotiation.cc:769] Sending connection context
0926 11:44:20.953760 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 11:44:20.953822 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":131,"threads_started":1}
I20260926 11:44:20.954557 26493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74417ef830f94e1186f6c39d10f0e039" candidate_uuid: "1d9c79a759a24889a719fdd27098fdaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20152320df954b4c889d1d6b505a953b" is_pre_election: true
I20260926 11:44:20.954547 26362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74417ef830f94e1186f6c39d10f0e039" candidate_uuid: "1d9c79a759a24889a719fdd27098fdaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7080253319db497cbf4e292cbe5245dd" is_pre_election: true
I20260926 11:44:20.954881 26362 raft_consensus.cc:2454] T 74417ef830f94e1186f6c39d10f0e039 P 7080253319db497cbf4e292cbe5245dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d9c79a759a24889a719fdd27098fdaf in term 0.
I20260926 11:44:20.954881 26493 raft_consensus.cc:2454] T 74417ef830f94e1186f6c39d10f0e039 P 20152320df954b4c889d1d6b505a953b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d9c79a759a24889a719fdd27098fdaf in term 0.
I20260926 11:44:20.955339 26557 leader_election.cc:302] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d9c79a759a24889a719fdd27098fdaf, 20152320df954b4c889d1d6b505a953b; no voters:
I20260926 11:44:20.955556 26695 raft_consensus.cc:2781] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 11:44:20.955633 26695 raft_consensus.cc:482] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 11:44:20.955693 26695 raft_consensus.cc:3037] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:20.956645 26695 raft_consensus.cc:504] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20152320df954b4c889d1d6b505a953b" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 43335 } } peers { permanent_uuid: "7080253319db497cbf4e292cbe5245dd" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 43929 } } peers { permanent_uuid: "1d9c79a759a24889a719fdd27098fdaf" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36997 } }
I20260926 11:44:20.956947 26695 leader_election.cc:288] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf [CANDIDATE]: Term 1 election: Requested vote from peers 20152320df954b4c889d1d6b505a953b (127.6.96.2:43335), 7080253319db497cbf4e292cbe5245dd (127.6.96.1:43929)
I20260926 11:44:20.957254 26362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74417ef830f94e1186f6c39d10f0e039" candidate_uuid: "1d9c79a759a24889a719fdd27098fdaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7080253319db497cbf4e292cbe5245dd"
I20260926 11:44:20.957252 26493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74417ef830f94e1186f6c39d10f0e039" candidate_uuid: "1d9c79a759a24889a719fdd27098fdaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20152320df954b4c889d1d6b505a953b"
I20260926 11:44:20.957362 26493 raft_consensus.cc:3037] T 74417ef830f94e1186f6c39d10f0e039 P 20152320df954b4c889d1d6b505a953b [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:20.957368 26362 raft_consensus.cc:3037] T 74417ef830f94e1186f6c39d10f0e039 P 7080253319db497cbf4e292cbe5245dd [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:20.958336 26362 raft_consensus.cc:2454] T 74417ef830f94e1186f6c39d10f0e039 P 7080253319db497cbf4e292cbe5245dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d9c79a759a24889a719fdd27098fdaf in term 1.
I20260926 11:44:20.958426 26493 raft_consensus.cc:2454] T 74417ef830f94e1186f6c39d10f0e039 P 20152320df954b4c889d1d6b505a953b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d9c79a759a24889a719fdd27098fdaf in term 1.
I20260926 11:44:20.958621 26560 leader_election.cc:302] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d9c79a759a24889a719fdd27098fdaf, 7080253319db497cbf4e292cbe5245dd; no voters:
I20260926 11:44:20.958770 26695 raft_consensus.cc:2781] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:20.958989 26695 raft_consensus.cc:686] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf [term 1 LEADER]: Becoming Leader. State: Replica: 1d9c79a759a24889a719fdd27098fdaf, State: Running, Role: LEADER
I20260926 11:44:20.959321 26695 consensus_queue.cc:237] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20152320df954b4c889d1d6b505a953b" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 43335 } } peers { permanent_uuid: "7080253319db497cbf4e292cbe5245dd" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 43929 } } peers { permanent_uuid: "1d9c79a759a24889a719fdd27098fdaf" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36997 } }
I20260926 11:44:20.962359 26220 catalog_manager.cc:5495] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d9c79a759a24889a719fdd27098fdaf (127.6.96.3). New cstate: current_term: 1 leader_uuid: "1d9c79a759a24889a719fdd27098fdaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20152320df954b4c889d1d6b505a953b" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 43335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7080253319db497cbf4e292cbe5245dd" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 43929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d9c79a759a24889a719fdd27098fdaf" member_type: VOTER last_known_addr { host: "127.6.96.3" port: 36997 } health_report { overall_health: HEALTHY } } }
I20260926 11:44:21.000272 26685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:20.997113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38390 (local address 127.6.96.3:36997)
0926 11:44:20.997223 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:20.997226 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:20.997239 (+ 13us) server_negotiation.cc:407] Connection header received
0926 11:44:20.997317 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:20.997319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:20.997373 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:20.997459 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:44:20.997679 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.998200 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.999070 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:20.999280 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:20.999356 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:20.999576 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:44:20.999991 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:44:21.000075 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:21.000099 (+ 24us) server_negotiation.cc:299] Negotiation successful
0926 11:44:21.000142 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 11:44:21.005837 26493 raft_consensus.cc:1261] T 74417ef830f94e1186f6c39d10f0e039 P 20152320df954b4c889d1d6b505a953b [term 1 FOLLOWER]: Refusing update from remote peer 1d9c79a759a24889a719fdd27098fdaf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:44:21.005836 26362 raft_consensus.cc:1261] T 74417ef830f94e1186f6c39d10f0e039 P 7080253319db497cbf4e292cbe5245dd [term 1 FOLLOWER]: Refusing update from remote peer 1d9c79a759a24889a719fdd27098fdaf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:44:21.006527 26697 consensus_queue.cc:1035] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "20152320df954b4c889d1d6b505a953b" member_type: VOTER last_known_addr { host: "127.6.96.2" port: 43335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:44:21.006707 26695 consensus_queue.cc:1035] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7080253319db497cbf4e292cbe5245dd" member_type: VOTER last_known_addr { host: "127.6.96.1" port: 43929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:44:21.019220 26701 mvcc.cc:205] Tried to move back new op lower bound from 7333572857873100800 to 7333572857691959296. Current Snapshot: MvccSnapshot[applied={T|T < 7333572857873100800 or (T in {7333572857873100800})}]
I20260926 11:44:21.020613 26218 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:44:21.020788 26218 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 11:44:21.023191 26218 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b41aaffa8eed4b8c98532c2cc3377ffa: Sending DeleteTablet for 3 replicas of tablet 74417ef830f94e1186f6c39d10f0e039
I20260926 11:44:21.024120 26473 tablet_service.cc:1514] Processing DeleteTablet for tablet 74417ef830f94e1186f6c39d10f0e039 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:44:21 UTC) from {username='slave'} at 127.0.0.1:48864
I20260926 11:44:21.024137 26604 tablet_service.cc:1514] Processing DeleteTablet for tablet 74417ef830f94e1186f6c39d10f0e039 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:44:21 UTC) from {username='slave'} at 127.0.0.1:38376
I20260926 11:44:21.024673 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 26280
I20260926 11:44:21.024714 26342 tablet_service.cc:1514] Processing DeleteTablet for tablet 74417ef830f94e1186f6c39d10f0e039 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 11:44:21 UTC) from {username='slave'} at 127.0.0.1:43786
I20260926 11:44:21.027211 26710 tablet_replica.cc:330] T 74417ef830f94e1186f6c39d10f0e039 P 20152320df954b4c889d1d6b505a953b: stopping tablet replica
I20260926 11:44:21.027537 26710 raft_consensus.cc:2229] T 74417ef830f94e1186f6c39d10f0e039 P 20152320df954b4c889d1d6b505a953b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 11:44:21.027860 26710 raft_consensus.cc:2258] T 74417ef830f94e1186f6c39d10f0e039 P 20152320df954b4c889d1d6b505a953b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:44:21.028244 26709 tablet_replica.cc:330] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf: stopping tablet replica
I20260926 11:44:21.028532 26709 raft_consensus.cc:2229] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf [term 1 LEADER]: Raft consensus shutting down.
I20260926 11:44:21.028911 26709 raft_consensus.cc:2258] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 11:44:21.030215 26710 ts_tablet_manager.cc:1905] T 74417ef830f94e1186f6c39d10f0e039 P 20152320df954b4c889d1d6b505a953b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 11:44:21.030215 26709 ts_tablet_manager.cc:1905] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 11:44:21.031160 26207 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.96.1:43929 (error 108)
I20260926 11:44:21.031945 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 26411
I20260926 11:44:21.033221 26709 ts_tablet_manager.cc:1918] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 11:44:21.033306 26709 log.cc:1196] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf: Deleting WAL directory at /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790422817500949-6528-0/minicluster-data/ts-2/wal/wals/74417ef830f94e1186f6c39d10f0e039
I20260926 11:44:21.033571 26709 ts_tablet_manager.cc:1939] T 74417ef830f94e1186f6c39d10f0e039 P 1d9c79a759a24889a719fdd27098fdaf: Deleting consensus metadata
I20260926 11:44:21.034613 26205 catalog_manager.cc:4841] TS 1d9c79a759a24889a719fdd27098fdaf (127.6.96.3:36997): tablet 74417ef830f94e1186f6c39d10f0e039 (table test-table [id=6647b26cb33a41a6866b5b2b20300225]) successfully deleted
I20260926 11:44:21.035904 26682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:21.035643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.1:43929 (local address 127.0.0.1:43794)
0926 11:44:21.035761 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:21.035825 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.1:43929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260926 11:44:21.036080 26207 catalog_manager.cc:4584] TS 7080253319db497cbf4e292cbe5245dd (127.6.96.1:43929): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 74417ef830f94e1186f6c39d10f0e039: Network error: Client connection negotiation failed: client connection to 127.6.96.1:43929: connect: Connection refused (error 111)
W20260926 11:44:21.039595 26682 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:21.039321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.2:43335 (local address 127.0.0.1:48876)
0926 11:44:21.039443 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:21.039457 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:21.039518 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.2:43335: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":39}
I20260926 11:44:21.039772 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 26542
I20260926 11:44:21.044504 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 26189
2026-09-26T11:44:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1071 ms)
[----------] 6 tests from EncryptionPolicyTest (10339 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:44:21 dist-test-slave-xxxx krb5kdc[26714](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:44:21 dist-test-slave-xxxx krb5kdc[26714](info): set up 2 sockets
Sep 26 11:44:21 dist-test-slave-xxxx krb5kdc[26714](info): commencing operation
krb5kdc: starting...
W20260926 11:44:22.010602 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.946s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:44:22 dist-test-slave-xxxx krb5kdc[26714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:44:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 11:44:22.129281 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37387
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:22.239428 26727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:22.239723 26727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:22.239782 26727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:22.243849 26727 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:22.243916 26727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:22.243942 26727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:22.243964 26727 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:22.243992 26727 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:22.249395 26727 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37387
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:22.250490 26727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:22.251564 26727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:22.259908 26732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:22.259912 26735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:22.260005 26727 server_base.cc:1034] running on GCE node
W20260926 11:44:22.259912 26733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:22.260759 26727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:22.261869 26727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:22.263022 26727 hybrid_clock.cc:648] HybridClock initialized: now 1790423062263004 us; error 33 us; skew 500 ppm
Sep 26 11:44:22 dist-test-slave-xxxx krb5kdc[26714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423062, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:22.265612 26727 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 11:44:22.266474 26727 webserver.cc:460] Webserver started at http://127.0.0.1:32971/ using document root <none> and password file <none>
I20260926 11:44:22.267056 26727 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:22.267118 26727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:22.267282 26727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:22.269250 26727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "a9552e3fd6214fcd9bf9acb6d2a52447"
format_stamp: "Formatted at 2026-09-26 11:44:22 on dist-test-slave-xxxx"
server_key: "2879458465992d9e142ceeba40462b5e"
server_key_iv: "89ed7ea584106a33718cc5272777b889"
server_key_version: "encryptionkey@0"
I20260926 11:44:22.269742 26727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "a9552e3fd6214fcd9bf9acb6d2a52447"
format_stamp: "Formatted at 2026-09-26 11:44:22 on dist-test-slave-xxxx"
server_key: "2879458465992d9e142ceeba40462b5e"
server_key_iv: "89ed7ea584106a33718cc5272777b889"
server_key_version: "encryptionkey@0"
I20260926 11:44:22.273281 26727 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 11:44:22.275673 26742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:22.276858 26727 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:44:22.276970 26727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "a9552e3fd6214fcd9bf9acb6d2a52447"
format_stamp: "Formatted at 2026-09-26 11:44:22 on dist-test-slave-xxxx"
server_key: "2879458465992d9e142ceeba40462b5e"
server_key_iv: "89ed7ea584106a33718cc5272777b889"
server_key_version: "encryptionkey@0"
I20260926 11:44:22.277070 26727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:22.290187 26727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:22.293066 26727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:22.293246 26727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:22.300038 26727 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37387
I20260926 11:44:22.300058 26794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37387 every 8 connection(s)
I20260926 11:44:22.300999 26727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:22.303776 26795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:22.305325 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 26727
I20260926 11:44:22.305466 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:22.305750 6528 external_mini_cluster.cc:1428] Setting key 02536fae4fb307b43e06c4906a6c0174
I20260926 11:44:22.310366 26795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9552e3fd6214fcd9bf9acb6d2a52447: Bootstrap starting.
I20260926 11:44:22.313139 26795 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9552e3fd6214fcd9bf9acb6d2a52447: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:22.314093 26795 log.cc:824] T 00000000000000000000000000000000 P a9552e3fd6214fcd9bf9acb6d2a52447: Log is configured to *not* fsync() on all Append() calls
Sep 26 11:44:22 dist-test-slave-xxxx krb5kdc[26714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 11:44:22.316480 26795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9552e3fd6214fcd9bf9acb6d2a52447: No bootstrap required, opened a new log
I20260926 11:44:22.320391 26798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:22.307158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34435 (local address 127.0.0.1:37387)
0926 11:44:22.307552 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:22.307569 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:22.307604 (+ 35us) server_negotiation.cc:407] Connection header received
0926 11:44:22.308225 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:22.308242 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:22.308612 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:22.308911 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:22.309793 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:22.310778 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:22.311915 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:22.312390 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:22.312500 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:22.315483 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:22.315489 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:22.315504 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:22.315531 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:22.317237 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:22.317816 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:22.317821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:22.317825 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:22.317941 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:22.318248 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:22.318250 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:22.318251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:22.318678 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:22.318896 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:22.319325 (+ 429us) server_negotiation.cc:299] Negotiation successful
0926 11:44:22.319573 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":111,"threads_started":1}
I20260926 11:44:22.320605 26795 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9552e3fd6214fcd9bf9acb6d2a52447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9552e3fd6214fcd9bf9acb6d2a52447" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37387 } }
I20260926 11:44:22.320851 26795 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9552e3fd6214fcd9bf9acb6d2a52447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:22.320907 26795 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9552e3fd6214fcd9bf9acb6d2a52447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9552e3fd6214fcd9bf9acb6d2a52447, State: Initialized, Role: FOLLOWER
I20260926 11:44:22.321349 26795 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9552e3fd6214fcd9bf9acb6d2a52447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9552e3fd6214fcd9bf9acb6d2a52447" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37387 } }
I20260926 11:44:22.321498 26795 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9552e3fd6214fcd9bf9acb6d2a52447 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:44:22.321566 26795 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9552e3fd6214fcd9bf9acb6d2a52447 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:44:22.321663 26795 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9552e3fd6214fcd9bf9acb6d2a52447 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:22.322774 26795 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9552e3fd6214fcd9bf9acb6d2a52447 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9552e3fd6214fcd9bf9acb6d2a52447" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37387 } }
I20260926 11:44:22.323181 26795 leader_election.cc:302] T 00000000000000000000000000000000 P a9552e3fd6214fcd9bf9acb6d2a52447 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9552e3fd6214fcd9bf9acb6d2a52447; no voters:
I20260926 11:44:22.323506 26795 leader_election.cc:288] T 00000000000000000000000000000000 P a9552e3fd6214fcd9bf9acb6d2a52447 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:44:22.323612 26800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9552e3fd6214fcd9bf9acb6d2a52447 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:22.323777 26800 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9552e3fd6214fcd9bf9acb6d2a52447 [term 1 LEADER]: Becoming Leader. State: Replica: a9552e3fd6214fcd9bf9acb6d2a52447, State: Running, Role: LEADER
I20260926 11:44:22.324091 26800 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9552e3fd6214fcd9bf9acb6d2a52447 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9552e3fd6214fcd9bf9acb6d2a52447" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37387 } }
I20260926 11:44:22.324199 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 11:44:22.324887 26795 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9552e3fd6214fcd9bf9acb6d2a52447 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:22.325889 26802 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9552e3fd6214fcd9bf9acb6d2a52447 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9552e3fd6214fcd9bf9acb6d2a52447. Latest consensus state: current_term: 1 leader_uuid: "a9552e3fd6214fcd9bf9acb6d2a52447" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9552e3fd6214fcd9bf9acb6d2a52447" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37387 } } }
I20260926 11:44:22.326028 26802 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9552e3fd6214fcd9bf9acb6d2a52447 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:22.326308 26801 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9552e3fd6214fcd9bf9acb6d2a52447 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9552e3fd6214fcd9bf9acb6d2a52447" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9552e3fd6214fcd9bf9acb6d2a52447" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37387 } } }
I20260926 11:44:22.326400 26801 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9552e3fd6214fcd9bf9acb6d2a52447 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:22.326858 26813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:22.330055 26813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:22.336195 26798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:22.326406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55051 (local address 127.0.0.1:37387)
0926 11:44:22.326596 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:22.326602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:22.327323 (+ 721us) server_negotiation.cc:407] Connection header received
0926 11:44:22.327466 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:22.327469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:22.327518 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:22.327596 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:22.328420 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:22.329155 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:22.330282 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:22.330603 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:22.330703 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:22.332448 (+ 1745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:22.332452 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:22.332455 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:22.332482 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:22.334085 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:22.334647 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:22.334652 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:22.334654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:22.334750 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:22.335242 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:22.335246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:22.335248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:22.335480 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:22.335631 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:22.335929 (+ 298us) server_negotiation.cc:299] Negotiation successful
0926 11:44:22.336033 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260926 11:44:22.336930 26813 catalog_manager.cc:1347] Generated new cluster ID: 51255447a46c49a798edd848074a047b
I20260926 11:44:22.336989 26813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:22.345774 26813 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:22.347196 26813 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:22.360674 26813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9552e3fd6214fcd9bf9acb6d2a52447: Generated new TSK 0
I20260926 11:44:22.361609 26813 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:44:22.368105 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 26727
W20260926 11:44:22.372748 26712 connection.cc:535] client connection to 127.0.0.1:37387 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37387): Connection reset by peer (error 104)
2026-09-26T11:44:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1322 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:44:22 dist-test-slave-xxxx krb5kdc[26826](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:44:22 dist-test-slave-xxxx krb5kdc[26826](info): set up 2 sockets
Sep 26 11:44:22 dist-test-slave-xxxx krb5kdc[26826](info): commencing operation
krb5kdc: starting...
W20260926 11:44:23.326783 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:44:23 dist-test-slave-xxxx krb5kdc[26826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:44:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 11:44:23.445506 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33861
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:23.552680 26839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:23.552944 26839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:23.552984 26839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:23.556932 26839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:23.556991 26839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:23.557009 26839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:23.557024 26839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:23.557037 26839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:23.562242 26839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33861
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:23.563282 26839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:23.564246 26839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:23.572672 26847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:23.572675 26845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:23.572726 26839 server_base.cc:1034] running on GCE node
W20260926 11:44:23.572750 26844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:23.573447 26839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:23.574580 26839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:23.575759 26839 hybrid_clock.cc:648] HybridClock initialized: now 1790423063575728 us; error 44 us; skew 500 ppm
Sep 26 11:44:23 dist-test-slave-xxxx krb5kdc[26826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423063, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:23.578410 26839 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 11:44:23.579350 26839 webserver.cc:460] Webserver started at http://127.0.0.1:41465/ using document root <none> and password file <none>
I20260926 11:44:23.579880 26839 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:23.579941 26839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:23.580147 26839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:23.582360 26839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "57047e97b79f4d03b5f43b3310da368b"
format_stamp: "Formatted at 2026-09-26 11:44:23 on dist-test-slave-xxxx"
server_key: "8594e0b7d74914d6130f8e6830a2c4c8"
server_key_iv: "954eaef4faa1216dc11a849f415d6504"
server_key_version: "encryptionkey@0"
I20260926 11:44:23.582959 26839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "57047e97b79f4d03b5f43b3310da368b"
format_stamp: "Formatted at 2026-09-26 11:44:23 on dist-test-slave-xxxx"
server_key: "8594e0b7d74914d6130f8e6830a2c4c8"
server_key_iv: "954eaef4faa1216dc11a849f415d6504"
server_key_version: "encryptionkey@0"
I20260926 11:44:23.586652 26839 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:44:23.589130 26854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:23.590404 26839 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:44:23.590525 26839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "57047e97b79f4d03b5f43b3310da368b"
format_stamp: "Formatted at 2026-09-26 11:44:23 on dist-test-slave-xxxx"
server_key: "8594e0b7d74914d6130f8e6830a2c4c8"
server_key_iv: "954eaef4faa1216dc11a849f415d6504"
server_key_version: "encryptionkey@0"
I20260926 11:44:23.590626 26839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:23.600327 26839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:23.603256 26839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:23.603436 26839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:23.610147 26839 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33861
I20260926 11:44:23.610163 26906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33861 every 8 connection(s)
I20260926 11:44:23.611158 26839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:23.613843 26907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:23.619226 26907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57047e97b79f4d03b5f43b3310da368b: Bootstrap starting.
I20260926 11:44:23.621008 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 26839
I20260926 11:44:23.621130 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:23.621407 6528 external_mini_cluster.cc:1428] Setting key afbeca9dfd633efc3925a4421a88eee2
I20260926 11:44:23.621702 26907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57047e97b79f4d03b5f43b3310da368b: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:23.622448 26907 log.cc:824] T 00000000000000000000000000000000 P 57047e97b79f4d03b5f43b3310da368b: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:23.624414 26907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57047e97b79f4d03b5f43b3310da368b: No bootstrap required, opened a new log
I20260926 11:44:23.627844 26907 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57047e97b79f4d03b5f43b3310da368b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57047e97b79f4d03b5f43b3310da368b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33861 } }
I20260926 11:44:23.628082 26907 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57047e97b79f4d03b5f43b3310da368b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:23.628149 26907 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57047e97b79f4d03b5f43b3310da368b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57047e97b79f4d03b5f43b3310da368b, State: Initialized, Role: FOLLOWER
I20260926 11:44:23.628551 26907 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57047e97b79f4d03b5f43b3310da368b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57047e97b79f4d03b5f43b3310da368b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33861 } }
I20260926 11:44:23.628671 26907 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57047e97b79f4d03b5f43b3310da368b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:44:23.628743 26907 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57047e97b79f4d03b5f43b3310da368b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:44:23.628837 26907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57047e97b79f4d03b5f43b3310da368b [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:23.629745 26907 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57047e97b79f4d03b5f43b3310da368b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57047e97b79f4d03b5f43b3310da368b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33861 } }
I20260926 11:44:23.629832 26910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:23.622468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48129 (local address 127.0.0.1:33861)
0926 11:44:23.622936 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:23.622955 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:23.623000 (+ 45us) server_negotiation.cc:407] Connection header received
0926 11:44:23.623713 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:23.623731 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:23.624067 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:23.624387 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:23.624791 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:23.625740 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:23.626619 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:23.627137 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:23.627291 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:23.628152 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:23.628158 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:23.628176 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:23.628616 (+ 440us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:23.628670 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:23.628701 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:23.628827 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:23.628994 (+ 167us) server_negotiation.cc:299] Negotiation successful
0926 11:44:23.629127 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":127,"threads_started":1}
I20260926 11:44:23.630090 26907 leader_election.cc:302] T 00000000000000000000000000000000 P 57047e97b79f4d03b5f43b3310da368b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57047e97b79f4d03b5f43b3310da368b; no voters:
I20260926 11:44:23.630431 26907 leader_election.cc:288] T 00000000000000000000000000000000 P 57047e97b79f4d03b5f43b3310da368b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:44:23.630509 26912 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57047e97b79f4d03b5f43b3310da368b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:23.630716 26912 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57047e97b79f4d03b5f43b3310da368b [term 1 LEADER]: Becoming Leader. State: Replica: 57047e97b79f4d03b5f43b3310da368b, State: Running, Role: LEADER
I20260926 11:44:23.631075 26912 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57047e97b79f4d03b5f43b3310da368b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57047e97b79f4d03b5f43b3310da368b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33861 } }
I20260926 11:44:23.631563 26907 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57047e97b79f4d03b5f43b3310da368b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:23.632284 26912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57047e97b79f4d03b5f43b3310da368b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57047e97b79f4d03b5f43b3310da368b. Latest consensus state: current_term: 1 leader_uuid: "57047e97b79f4d03b5f43b3310da368b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57047e97b79f4d03b5f43b3310da368b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33861 } } }
I20260926 11:44:23.632470 26912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57047e97b79f4d03b5f43b3310da368b [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:23.632679 26913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57047e97b79f4d03b5f43b3310da368b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57047e97b79f4d03b5f43b3310da368b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57047e97b79f4d03b5f43b3310da368b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33861 } } }
I20260926 11:44:23.632771 26913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57047e97b79f4d03b5f43b3310da368b [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:23.632910 26921 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:23.632917 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 11:44:23.636219 26921 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:23.639819 26910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:23.635205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60547 (local address 127.0.0.1:33861)
0926 11:44:23.635315 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:23.635320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:23.635665 (+ 345us) server_negotiation.cc:407] Connection header received
0926 11:44:23.635784 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:23.635787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:23.635834 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:23.635913 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:23.636189 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:23.636936 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:23.638015 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:23.638372 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:23.638476 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:23.639355 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:23.639359 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:23.639362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:23.639375 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:23.639402 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:23.639416 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:23.639505 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:23.639653 (+ 148us) server_negotiation.cc:299] Negotiation successful
0926 11:44:23.639688 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 11:44:23.641876 26921 catalog_manager.cc:1347] Generated new cluster ID: d9fa5633900642f29a080780c844bfee
I20260926 11:44:23.641937 26921 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:23.661221 26921 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:23.662487 26921 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:23.669662 26921 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57047e97b79f4d03b5f43b3310da368b: Generated new TSK 0
I20260926 11:44:23.670372 26921 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:44:23.681107 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 26839
W20260926 11:44:23.687409 26824 connection.cc:535] client connection to 127.0.0.1:33861 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33861): Connection reset by peer (error 104)
2026-09-26T11:44:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1314 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:44:23 dist-test-slave-xxxx krb5kdc[26938](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:44:23 dist-test-slave-xxxx krb5kdc[26938](info): set up 2 sockets
Sep 26 11:44:23 dist-test-slave-xxxx krb5kdc[26938](info): commencing operation
krb5kdc: starting...
W20260926 11:44:24.630893 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:44:24 dist-test-slave-xxxx krb5kdc[26938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:44:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 11:44:24.753585 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36389
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:24.866983 26951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:24.867271 26951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:24.867313 26951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:24.871500 26951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:24.871572 26951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:24.871590 26951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:24.871608 26951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:24.871644 26951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:24.877080 26951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36389
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:24.878222 26951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:24.879320 26951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:24.888016 26956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:24.888022 26957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:24.888135 26951 server_base.cc:1034] running on GCE node
W20260926 11:44:24.888016 26959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:24.888867 26951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:24.890116 26951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:24.891314 26951 hybrid_clock.cc:648] HybridClock initialized: now 1790423064891280 us; error 55 us; skew 500 ppm
Sep 26 11:44:24 dist-test-slave-xxxx krb5kdc[26938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423064, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:24.901172 26951 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 11:44:24.902251 26951 webserver.cc:460] Webserver started at http://127.0.0.1:42499/ using document root <none> and password file <none>
I20260926 11:44:24.902822 26951 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:24.902945 26951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:24.903159 26951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:24.905227 26951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "a8f578fa1a2844b19be6fb1dd7fde873"
format_stamp: "Formatted at 2026-09-26 11:44:24 on dist-test-slave-xxxx"
server_key: "49e6f0ab3db459bb3054ec7533069954"
server_key_iv: "556e2bba99084a694574190e1ed5599a"
server_key_version: "encryptionkey@0"
I20260926 11:44:24.905762 26951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "a8f578fa1a2844b19be6fb1dd7fde873"
format_stamp: "Formatted at 2026-09-26 11:44:24 on dist-test-slave-xxxx"
server_key: "49e6f0ab3db459bb3054ec7533069954"
server_key_iv: "556e2bba99084a694574190e1ed5599a"
server_key_version: "encryptionkey@0"
I20260926 11:44:24.909463 26951 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260926 11:44:24.911867 26966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:24.913021 26951 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:44:24.913132 26951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "a8f578fa1a2844b19be6fb1dd7fde873"
format_stamp: "Formatted at 2026-09-26 11:44:24 on dist-test-slave-xxxx"
server_key: "49e6f0ab3db459bb3054ec7533069954"
server_key_iv: "556e2bba99084a694574190e1ed5599a"
server_key_version: "encryptionkey@0"
I20260926 11:44:24.913223 26951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:24.917629 26951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:24.920576 26951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:24.920735 26951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:24.927193 26951 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36389
I20260926 11:44:24.927214 27018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36389 every 8 connection(s)
I20260926 11:44:24.928068 26951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:24.929828 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 26951
I20260926 11:44:24.929960 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:24.930209 6528 external_mini_cluster.cc:1428] Setting key 63ccda81179e73911a7ec65f192cb37e
I20260926 11:44:24.930989 27019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:24.935184 27022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:24.931486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37149 (local address 127.0.0.1:36389)
0926 11:44:24.931862 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:24.931878 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:24.931914 (+ 36us) server_negotiation.cc:407] Connection header received
0926 11:44:24.932538 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:24.932554 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:24.932828 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:24.933108 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:24.933939 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:24.933944 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:24.933956 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:24.933967 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:24.934019 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:24.934038 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:24.934148 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:24.934247 (+ 99us) server_negotiation.cc:299] Negotiation successful
0926 11:44:24.934395 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":102,"threads_started":1}
I20260926 11:44:24.937489 27019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8f578fa1a2844b19be6fb1dd7fde873: Bootstrap starting.
I20260926 11:44:24.938097 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 11:44:24.939826 27019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8f578fa1a2844b19be6fb1dd7fde873: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:24.940613 27019 log.cc:824] T 00000000000000000000000000000000 P a8f578fa1a2844b19be6fb1dd7fde873: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:24.941874 27022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:24.940211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40887 (local address 127.0.0.1:36389)
0926 11:44:24.940341 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:24.940345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:24.940557 (+ 212us) server_negotiation.cc:407] Connection header received
0926 11:44:24.940647 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:24.940650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:24.940690 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:24.940773 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:24.941435 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:24.941439 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:24.941442 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:24.941453 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:24.941479 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:24.941492 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:24.941576 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:24.941698 (+ 122us) server_negotiation.cc:299] Negotiation successful
0926 11:44:24.941734 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 11:44:24.942701 27019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8f578fa1a2844b19be6fb1dd7fde873: No bootstrap required, opened a new log
I20260926 11:44:24.946131 27019 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8f578fa1a2844b19be6fb1dd7fde873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f578fa1a2844b19be6fb1dd7fde873" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36389 } }
I20260926 11:44:24.946369 27019 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8f578fa1a2844b19be6fb1dd7fde873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:24.946425 27019 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8f578fa1a2844b19be6fb1dd7fde873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8f578fa1a2844b19be6fb1dd7fde873, State: Initialized, Role: FOLLOWER
I20260926 11:44:24.946880 27019 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8f578fa1a2844b19be6fb1dd7fde873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f578fa1a2844b19be6fb1dd7fde873" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36389 } }
I20260926 11:44:24.947036 27019 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8f578fa1a2844b19be6fb1dd7fde873 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:44:24.947099 27019 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8f578fa1a2844b19be6fb1dd7fde873 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:44:24.947192 27019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8f578fa1a2844b19be6fb1dd7fde873 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:24.948092 27019 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8f578fa1a2844b19be6fb1dd7fde873 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f578fa1a2844b19be6fb1dd7fde873" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36389 } }
I20260926 11:44:24.948426 27019 leader_election.cc:302] T 00000000000000000000000000000000 P a8f578fa1a2844b19be6fb1dd7fde873 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8f578fa1a2844b19be6fb1dd7fde873; no voters:
I20260926 11:44:24.948757 27019 leader_election.cc:288] T 00000000000000000000000000000000 P a8f578fa1a2844b19be6fb1dd7fde873 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:44:24.948855 27029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8f578fa1a2844b19be6fb1dd7fde873 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:24.949060 27029 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8f578fa1a2844b19be6fb1dd7fde873 [term 1 LEADER]: Becoming Leader. State: Replica: a8f578fa1a2844b19be6fb1dd7fde873, State: Running, Role: LEADER
I20260926 11:44:24.949397 27029 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8f578fa1a2844b19be6fb1dd7fde873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f578fa1a2844b19be6fb1dd7fde873" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36389 } }
I20260926 11:44:24.949932 27019 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8f578fa1a2844b19be6fb1dd7fde873 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:24.950963 27030 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8f578fa1a2844b19be6fb1dd7fde873 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8f578fa1a2844b19be6fb1dd7fde873" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f578fa1a2844b19be6fb1dd7fde873" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36389 } } }
I20260926 11:44:24.951089 27030 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8f578fa1a2844b19be6fb1dd7fde873 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:24.951383 27031 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8f578fa1a2844b19be6fb1dd7fde873 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8f578fa1a2844b19be6fb1dd7fde873. Latest consensus state: current_term: 1 leader_uuid: "a8f578fa1a2844b19be6fb1dd7fde873" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f578fa1a2844b19be6fb1dd7fde873" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36389 } } }
I20260926 11:44:24.951432 27038 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:24.951543 27031 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8f578fa1a2844b19be6fb1dd7fde873 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:24.954212 27038 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:24.960270 27038 catalog_manager.cc:1347] Generated new cluster ID: 87ddf8aedc594667be0c11bb5ddde835
I20260926 11:44:24.960354 27038 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:24.984161 27038 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:24.984616 27038 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:24.997390 27038 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8f578fa1a2844b19be6fb1dd7fde873: Generated new TSK 0
I20260926 11:44:24.998086 27038 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:44:25.001389 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 26951
2026-09-26T11:44:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1318 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:44:25 dist-test-slave-xxxx krb5kdc[27050](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:44:25 dist-test-slave-xxxx krb5kdc[27050](info): set up 2 sockets
Sep 26 11:44:25 dist-test-slave-xxxx krb5kdc[27050](info): commencing operation
krb5kdc: starting...
W20260926 11:44:25.950680 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.931s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:44:26 dist-test-slave-xxxx krb5kdc[27050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:44:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 11:44:26.069185 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39057
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:26.177199 27063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:26.177487 27063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:26.177529 27063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:26.181474 27063 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:26.181540 27063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:26.181558 27063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:26.181574 27063 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:26.181588 27063 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:26.186851 27063 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:26.187862 27063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:26.188835 27063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:26.197537 27069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:26.197643 27068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:26.197714 27063 server_base.cc:1034] running on GCE node
W20260926 11:44:26.197535 27071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:26.198451 27063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:26.199630 27063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:26.200830 27063 hybrid_clock.cc:648] HybridClock initialized: now 1790423066200810 us; error 33 us; skew 500 ppm
Sep 26 11:44:26 dist-test-slave-xxxx krb5kdc[27050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423066, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:26.203753 27063 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 11:44:26.204742 27063 webserver.cc:460] Webserver started at http://127.0.0.1:33753/ using document root <none> and password file <none>
I20260926 11:44:26.205302 27063 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:26.205368 27063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:26.205565 27063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:26.207674 27063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "7e85463f8e5847cc8dd9c9796dd40721"
format_stamp: "Formatted at 2026-09-26 11:44:26 on dist-test-slave-xxxx"
server_key: "62fe997c5d7a00f80879b1defd5e92ff"
server_key_iv: "166fa24f622b97e01f73afcacdee5af4"
server_key_version: "encryptionkey@0"
I20260926 11:44:26.208204 27063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "7e85463f8e5847cc8dd9c9796dd40721"
format_stamp: "Formatted at 2026-09-26 11:44:26 on dist-test-slave-xxxx"
server_key: "62fe997c5d7a00f80879b1defd5e92ff"
server_key_iv: "166fa24f622b97e01f73afcacdee5af4"
server_key_version: "encryptionkey@0"
I20260926 11:44:26.211736 27063 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 11:44:26.214058 27078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:26.215324 27063 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:44:26.215436 27063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "7e85463f8e5847cc8dd9c9796dd40721"
format_stamp: "Formatted at 2026-09-26 11:44:26 on dist-test-slave-xxxx"
server_key: "62fe997c5d7a00f80879b1defd5e92ff"
server_key_iv: "166fa24f622b97e01f73afcacdee5af4"
server_key_version: "encryptionkey@0"
I20260926 11:44:26.215530 27063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:26.258347 27063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:26.261479 27063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:26.261675 27063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:26.268481 27063 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39057
I20260926 11:44:26.268492 27130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39057 every 8 connection(s)
I20260926 11:44:26.269433 27063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:26.272248 27131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:26.275332 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 27063
I20260926 11:44:26.275466 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:26.275734 6528 external_mini_cluster.cc:1428] Setting key 48d4b35677502ad222539bf4d774b8d5
I20260926 11:44:26.278335 27131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e85463f8e5847cc8dd9c9796dd40721: Bootstrap starting.
I20260926 11:44:26.281119 27131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e85463f8e5847cc8dd9c9796dd40721: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:26.281994 27131 log.cc:824] T 00000000000000000000000000000000 P 7e85463f8e5847cc8dd9c9796dd40721: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:26.284258 27131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e85463f8e5847cc8dd9c9796dd40721: No bootstrap required, opened a new log
Sep 26 11:44:26 dist-test-slave-xxxx krb5kdc[27050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 11:44:26.288008 27131 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e85463f8e5847cc8dd9c9796dd40721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e85463f8e5847cc8dd9c9796dd40721" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39057 } }
I20260926 11:44:26.288317 27131 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e85463f8e5847cc8dd9c9796dd40721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:26.288376 27131 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e85463f8e5847cc8dd9c9796dd40721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e85463f8e5847cc8dd9c9796dd40721, State: Initialized, Role: FOLLOWER
I20260926 11:44:26.288889 27131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e85463f8e5847cc8dd9c9796dd40721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e85463f8e5847cc8dd9c9796dd40721" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39057 } }
I20260926 11:44:26.289040 27131 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e85463f8e5847cc8dd9c9796dd40721 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:44:26.289112 27131 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e85463f8e5847cc8dd9c9796dd40721 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:44:26.289217 27131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e85463f8e5847cc8dd9c9796dd40721 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:26.290372 27131 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e85463f8e5847cc8dd9c9796dd40721 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e85463f8e5847cc8dd9c9796dd40721" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39057 } }
I20260926 11:44:26.290767 27131 leader_election.cc:302] T 00000000000000000000000000000000 P 7e85463f8e5847cc8dd9c9796dd40721 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e85463f8e5847cc8dd9c9796dd40721; no voters:
I20260926 11:44:26.291203 27131 leader_election.cc:288] T 00000000000000000000000000000000 P 7e85463f8e5847cc8dd9c9796dd40721 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:44:26.291325 27136 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e85463f8e5847cc8dd9c9796dd40721 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:26.291576 27136 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e85463f8e5847cc8dd9c9796dd40721 [term 1 LEADER]: Becoming Leader. State: Replica: 7e85463f8e5847cc8dd9c9796dd40721, State: Running, Role: LEADER
I20260926 11:44:26.291940 27136 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e85463f8e5847cc8dd9c9796dd40721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e85463f8e5847cc8dd9c9796dd40721" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39057 } }
I20260926 11:44:26.292059 27134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:26.277187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37179 (local address 127.0.0.1:39057)
0926 11:44:26.277627 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:26.277647 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:26.277691 (+ 44us) server_negotiation.cc:407] Connection header received
0926 11:44:26.278514 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:26.278538 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:26.279014 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:26.279400 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:26.280089 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:26.281612 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:26.282665 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:26.283348 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:26.283517 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:26.285909 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:26.285916 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:26.285935 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:26.285970 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:26.288642 (+ 2672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:26.289089 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:26.289095 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:26.289102 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:26.289250 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:26.289539 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:26.289542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:26.289544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:26.290095 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:26.290313 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:26.290582 (+ 269us) server_negotiation.cc:299] Negotiation successful
0926 11:44:26.290932 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":119,"threads_started":1}
I20260926 11:44:26.292517 27131 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e85463f8e5847cc8dd9c9796dd40721 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:26.293705 27138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e85463f8e5847cc8dd9c9796dd40721 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e85463f8e5847cc8dd9c9796dd40721. Latest consensus state: current_term: 1 leader_uuid: "7e85463f8e5847cc8dd9c9796dd40721" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e85463f8e5847cc8dd9c9796dd40721" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39057 } } }
I20260926 11:44:26.293705 27137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e85463f8e5847cc8dd9c9796dd40721 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e85463f8e5847cc8dd9c9796dd40721" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e85463f8e5847cc8dd9c9796dd40721" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39057 } } }
I20260926 11:44:26.293848 27137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e85463f8e5847cc8dd9c9796dd40721 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:26.293848 27138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e85463f8e5847cc8dd9c9796dd40721 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:26.295188 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260926 11:44:26.297402 27151 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7e85463f8e5847cc8dd9c9796dd40721: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 11:44:26.297603 27151 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 11:44:26.297658 27145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:26.298535 27145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:26.304984 27134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:26.297395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47403 (local address 127.0.0.1:39057)
0926 11:44:26.297509 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:26.297514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:26.297765 (+ 251us) server_negotiation.cc:407] Connection header received
0926 11:44:26.297882 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:26.297885 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:26.297927 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:26.297996 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:26.298632 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:26.299269 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:26.300071 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:26.300304 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:26.300390 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:26.301656 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:26.301660 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:26.301663 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:26.301688 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:26.303137 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:26.303636 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:26.303640 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:26.303641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:26.303729 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:26.304026 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:26.304029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:26.304031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:26.304253 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:26.304377 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:26.304689 (+ 312us) server_negotiation.cc:299] Negotiation successful
0926 11:44:26.304819 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 11:44:26.305202 27145 catalog_manager.cc:1347] Generated new cluster ID: 611a6a9018cf4a208e5d6ecb42009dd5
I20260926 11:44:26.305275 27145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:26.320616 27145 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:26.321557 27145 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:26.326109 27145 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e85463f8e5847cc8dd9c9796dd40721: Generated new TSK 0
I20260926 11:44:26.326793 27145 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:44:26.331451 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 27063
2026-09-26T11:44:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1329 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:44:26 dist-test-slave-xxxx krb5kdc[27162](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:44:26 dist-test-slave-xxxx krb5kdc[27162](info): set up 2 sockets
Sep 26 11:44:26 dist-test-slave-xxxx krb5kdc[27162](info): commencing operation
krb5kdc: starting...
W20260926 11:44:27.282817 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:44:27 dist-test-slave-xxxx krb5kdc[27162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:44:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 11:44:27.402191 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42193
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:27.511130 27175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:27.511413 27175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:27.511456 27175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:27.515509 27175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:27.515571 27175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:27.515589 27175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:27.515604 27175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:27.515618 27175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:27.521051 27175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:27.522097 27175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:27.523102 27175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 11:44:27.531461 27175 server_base.cc:1034] running on GCE node
W20260926 11:44:27.531481 27181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:27.531492 27180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:27.531704 27183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:27.532262 27175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:27.533390 27175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:27.534564 27175 hybrid_clock.cc:648] HybridClock initialized: now 1790423067534547 us; error 29 us; skew 500 ppm
Sep 26 11:44:27 dist-test-slave-xxxx krb5kdc[27162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423067, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:27.541391 27175 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 11:44:27.542304 27175 webserver.cc:460] Webserver started at http://127.0.0.1:37067/ using document root <none> and password file <none>
I20260926 11:44:27.542822 27175 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:27.542922 27175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:27.543110 27175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:27.545198 27175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "fff551afb63d4e6cab1e53a84ae7b250"
format_stamp: "Formatted at 2026-09-26 11:44:27 on dist-test-slave-xxxx"
server_key: "1caf0da8a8101600145aee0507eaa4da"
server_key_iv: "7ec141bbd0b00dfbe32b51cd4de57c98"
server_key_version: "encryptionkey@0"
I20260926 11:44:27.545899 27175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "fff551afb63d4e6cab1e53a84ae7b250"
format_stamp: "Formatted at 2026-09-26 11:44:27 on dist-test-slave-xxxx"
server_key: "1caf0da8a8101600145aee0507eaa4da"
server_key_iv: "7ec141bbd0b00dfbe32b51cd4de57c98"
server_key_version: "encryptionkey@0"
I20260926 11:44:27.551139 27175 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 11:44:27.554548 27190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:27.556392 27175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:44:27.556547 27175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "fff551afb63d4e6cab1e53a84ae7b250"
format_stamp: "Formatted at 2026-09-26 11:44:27 on dist-test-slave-xxxx"
server_key: "1caf0da8a8101600145aee0507eaa4da"
server_key_iv: "7ec141bbd0b00dfbe32b51cd4de57c98"
server_key_version: "encryptionkey@0"
I20260926 11:44:27.556851 27175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:27.580024 27175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:27.583376 27175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:27.583561 27175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:27.591284 27175 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42193
I20260926 11:44:27.591305 27242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42193 every 8 connection(s)
I20260926 11:44:27.592212 27175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:27.594981 27243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:27.598407 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 27175
I20260926 11:44:27.598549 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:27.598783 6528 external_mini_cluster.cc:1428] Setting key 36852782823a3c2a3e70c42f2dc08ef0
I20260926 11:44:27.600751 27243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fff551afb63d4e6cab1e53a84ae7b250: Bootstrap starting.
I20260926 11:44:27.603046 27243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fff551afb63d4e6cab1e53a84ae7b250: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:27.603838 27243 log.cc:824] T 00000000000000000000000000000000 P fff551afb63d4e6cab1e53a84ae7b250: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:27.605955 27243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fff551afb63d4e6cab1e53a84ae7b250: No bootstrap required, opened a new log
I20260926 11:44:27.607920 27246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:27.600227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34841 (local address 127.0.0.1:42193)
0926 11:44:27.600634 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:27.600649 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:27.600684 (+ 35us) server_negotiation.cc:407] Connection header received
0926 11:44:27.601315 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:27.601332 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:27.601686 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:27.601955 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:27.602316 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:27.603417 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:27.604540 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:27.605055 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:27.605170 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:27.605963 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:27.605968 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:27.605980 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:27.606400 (+ 420us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:27.606453 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:27.606497 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:27.606573 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:27.606775 (+ 202us) server_negotiation.cc:299] Negotiation successful
0926 11:44:27.607022 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":108,"threads_started":1}
I20260926 11:44:27.609225 27243 raft_consensus.cc:348] T 00000000000000000000000000000000 P fff551afb63d4e6cab1e53a84ae7b250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff551afb63d4e6cab1e53a84ae7b250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42193 } }
I20260926 11:44:27.609494 27243 raft_consensus.cc:374] T 00000000000000000000000000000000 P fff551afb63d4e6cab1e53a84ae7b250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:27.609551 27243 raft_consensus.cc:729] T 00000000000000000000000000000000 P fff551afb63d4e6cab1e53a84ae7b250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fff551afb63d4e6cab1e53a84ae7b250, State: Initialized, Role: FOLLOWER
I20260926 11:44:27.609980 27243 consensus_queue.cc:260] T 00000000000000000000000000000000 P fff551afb63d4e6cab1e53a84ae7b250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff551afb63d4e6cab1e53a84ae7b250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42193 } }
I20260926 11:44:27.610127 27243 raft_consensus.cc:388] T 00000000000000000000000000000000 P fff551afb63d4e6cab1e53a84ae7b250 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:44:27.610195 27243 raft_consensus.cc:482] T 00000000000000000000000000000000 P fff551afb63d4e6cab1e53a84ae7b250 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:44:27.610294 27243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fff551afb63d4e6cab1e53a84ae7b250 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:27.610764 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 11:44:27.611313 27243 raft_consensus.cc:504] T 00000000000000000000000000000000 P fff551afb63d4e6cab1e53a84ae7b250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff551afb63d4e6cab1e53a84ae7b250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42193 } }
I20260926 11:44:27.611631 27243 leader_election.cc:302] T 00000000000000000000000000000000 P fff551afb63d4e6cab1e53a84ae7b250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fff551afb63d4e6cab1e53a84ae7b250; no voters:
I20260926 11:44:27.611912 27243 leader_election.cc:288] T 00000000000000000000000000000000 P fff551afb63d4e6cab1e53a84ae7b250 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:44:27.611989 27250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fff551afb63d4e6cab1e53a84ae7b250 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:27.612182 27250 raft_consensus.cc:686] T 00000000000000000000000000000000 P fff551afb63d4e6cab1e53a84ae7b250 [term 1 LEADER]: Becoming Leader. State: Replica: fff551afb63d4e6cab1e53a84ae7b250, State: Running, Role: LEADER
I20260926 11:44:27.612915 27250 consensus_queue.cc:237] T 00000000000000000000000000000000 P fff551afb63d4e6cab1e53a84ae7b250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff551afb63d4e6cab1e53a84ae7b250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42193 } }
I20260926 11:44:27.613157 27243 sys_catalog.cc:564] T 00000000000000000000000000000000 P fff551afb63d4e6cab1e53a84ae7b250 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:27.614063 27250 sys_catalog.cc:455] T 00000000000000000000000000000000 P fff551afb63d4e6cab1e53a84ae7b250 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fff551afb63d4e6cab1e53a84ae7b250. Latest consensus state: current_term: 1 leader_uuid: "fff551afb63d4e6cab1e53a84ae7b250" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff551afb63d4e6cab1e53a84ae7b250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42193 } } }
I20260926 11:44:27.614178 27250 sys_catalog.cc:458] T 00000000000000000000000000000000 P fff551afb63d4e6cab1e53a84ae7b250 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:27.614526 27262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:27.614545 27253 sys_catalog.cc:455] T 00000000000000000000000000000000 P fff551afb63d4e6cab1e53a84ae7b250 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fff551afb63d4e6cab1e53a84ae7b250" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fff551afb63d4e6cab1e53a84ae7b250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42193 } } }
I20260926 11:44:27.614715 27253 sys_catalog.cc:458] T 00000000000000000000000000000000 P fff551afb63d4e6cab1e53a84ae7b250 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:27.619150 27262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:27.620004 27246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:27.612740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58551 (local address 127.0.0.1:42193)
0926 11:44:27.612850 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:27.612854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:27.613002 (+ 148us) server_negotiation.cc:407] Connection header received
0926 11:44:27.613165 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:27.613169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:27.613236 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:27.613316 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:27.613583 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:27.614443 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:27.615633 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:27.615977 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:27.616070 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:27.619404 (+ 3334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:27.619408 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:27.619411 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:27.619423 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:27.619453 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:27.619465 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:27.619524 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:27.619841 (+ 317us) server_negotiation.cc:299] Negotiation successful
0926 11:44:27.619897 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":40}
I20260926 11:44:27.625043 27262 catalog_manager.cc:1347] Generated new cluster ID: 58b66364bd6f465185be1b97564086da
I20260926 11:44:27.625097 27262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:27.631474 27262 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:27.632642 27262 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:27.654090 27262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fff551afb63d4e6cab1e53a84ae7b250: Generated new TSK 0
I20260926 11:44:27.654904 27262 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:44:27.662385 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 27175
2026-09-26T11:44:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1331 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:44:27 dist-test-slave-xxxx krb5kdc[27274](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:44:27 dist-test-slave-xxxx krb5kdc[27274](info): set up 2 sockets
Sep 26 11:44:27 dist-test-slave-xxxx krb5kdc[27274](info): commencing operation
krb5kdc: starting...
W20260926 11:44:28.630769 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.950s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:44:28 dist-test-slave-xxxx krb5kdc[27274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:44:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 11:44:28.749810 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42771
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:28.854683 27287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:28.854944 27287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:28.854984 27287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:28.858819 27287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:28.858888 27287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:28.858906 27287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:28.858920 27287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:28.858933 27287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:28.863962 27287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:28.864899 27287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:28.865815 27287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:28.873818 27292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:28.873874 27287 server_base.cc:1034] running on GCE node
W20260926 11:44:28.873812 27295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:28.873812 27293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:28.874502 27287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:28.875559 27287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:28.876708 27287 hybrid_clock.cc:648] HybridClock initialized: now 1790423068876693 us; error 28 us; skew 500 ppm
Sep 26 11:44:28 dist-test-slave-xxxx krb5kdc[27274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423068, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:28.879184 27287 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 11:44:28.880002 27287 webserver.cc:460] Webserver started at http://127.0.0.1:34347/ using document root <none> and password file <none>
I20260926 11:44:28.880504 27287 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:28.880561 27287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:28.880774 27287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:28.882699 27287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "89158b19e9c642fcab8a1c51b4bc4bc9"
format_stamp: "Formatted at 2026-09-26 11:44:28 on dist-test-slave-xxxx"
server_key: "f3060bd059d76c34569e4a12626d14c3"
server_key_iv: "5e975020335b7a207ace9ef1e21c2126"
server_key_version: "encryptionkey@0"
I20260926 11:44:28.883247 27287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "89158b19e9c642fcab8a1c51b4bc4bc9"
format_stamp: "Formatted at 2026-09-26 11:44:28 on dist-test-slave-xxxx"
server_key: "f3060bd059d76c34569e4a12626d14c3"
server_key_iv: "5e975020335b7a207ace9ef1e21c2126"
server_key_version: "encryptionkey@0"
I20260926 11:44:28.886730 27287 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:44:28.889103 27302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:28.890290 27287 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:44:28.890390 27287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "89158b19e9c642fcab8a1c51b4bc4bc9"
format_stamp: "Formatted at 2026-09-26 11:44:28 on dist-test-slave-xxxx"
server_key: "f3060bd059d76c34569e4a12626d14c3"
server_key_iv: "5e975020335b7a207ace9ef1e21c2126"
server_key_version: "encryptionkey@0"
I20260926 11:44:28.890489 27287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:28.894759 27287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:28.897387 27287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:28.897540 27287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:28.904032 27287 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42771
I20260926 11:44:28.904042 27354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42771 every 8 connection(s)
I20260926 11:44:28.904904 27287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:28.907547 27355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:28.912858 27355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89158b19e9c642fcab8a1c51b4bc4bc9: Bootstrap starting.
I20260926 11:44:28.915107 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 27287
I20260926 11:44:28.915242 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:28.915310 27355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89158b19e9c642fcab8a1c51b4bc4bc9: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:28.915531 6528 external_mini_cluster.cc:1428] Setting key d92c21fa73fd461e7cb4603848473ee9
I20260926 11:44:28.916121 27355 log.cc:824] T 00000000000000000000000000000000 P 89158b19e9c642fcab8a1c51b4bc4bc9: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:28.917981 27355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89158b19e9c642fcab8a1c51b4bc4bc9: No bootstrap required, opened a new log
I20260926 11:44:28.920161 27359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:28.916591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34191 (local address 127.0.0.1:42771)
0926 11:44:28.916953 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:28.916967 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:28.917003 (+ 36us) server_negotiation.cc:407] Connection header received
0926 11:44:28.917644 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:28.917660 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:28.917930 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:28.918260 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:28.919024 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:28.919028 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:28.919039 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:28.919049 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:28.919105 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:28.919124 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:28.919260 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:28.919348 (+ 88us) server_negotiation.cc:299] Negotiation successful
0926 11:44:28.919493 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":78,"threads_started":1}
I20260926 11:44:28.921769 27355 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89158b19e9c642fcab8a1c51b4bc4bc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89158b19e9c642fcab8a1c51b4bc4bc9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42771 } }
I20260926 11:44:28.922024 27355 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89158b19e9c642fcab8a1c51b4bc4bc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:28.922080 27355 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89158b19e9c642fcab8a1c51b4bc4bc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89158b19e9c642fcab8a1c51b4bc4bc9, State: Initialized, Role: FOLLOWER
I20260926 11:44:28.922518 27355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89158b19e9c642fcab8a1c51b4bc4bc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89158b19e9c642fcab8a1c51b4bc4bc9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42771 } }
I20260926 11:44:28.922678 27355 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89158b19e9c642fcab8a1c51b4bc4bc9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:44:28.922744 27355 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89158b19e9c642fcab8a1c51b4bc4bc9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:44:28.922864 27355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89158b19e9c642fcab8a1c51b4bc4bc9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:28.922915 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 11:44:28.923707 27355 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89158b19e9c642fcab8a1c51b4bc4bc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89158b19e9c642fcab8a1c51b4bc4bc9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42771 } }
I20260926 11:44:28.924019 27355 leader_election.cc:302] T 00000000000000000000000000000000 P 89158b19e9c642fcab8a1c51b4bc4bc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89158b19e9c642fcab8a1c51b4bc4bc9; no voters:
I20260926 11:44:28.924315 27355 leader_election.cc:288] T 00000000000000000000000000000000 P 89158b19e9c642fcab8a1c51b4bc4bc9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:44:28.924418 27364 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89158b19e9c642fcab8a1c51b4bc4bc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:28.924636 27364 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89158b19e9c642fcab8a1c51b4bc4bc9 [term 1 LEADER]: Becoming Leader. State: Replica: 89158b19e9c642fcab8a1c51b4bc4bc9, State: Running, Role: LEADER
I20260926 11:44:28.924976 27364 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89158b19e9c642fcab8a1c51b4bc4bc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89158b19e9c642fcab8a1c51b4bc4bc9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42771 } }
I20260926 11:44:28.925575 27355 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89158b19e9c642fcab8a1c51b4bc4bc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:28.926502 27359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:28.925098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37083 (local address 127.0.0.1:42771)
0926 11:44:28.925204 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:28.925207 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:28.925220 (+ 13us) server_negotiation.cc:407] Connection header received
0926 11:44:28.925253 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:28.925256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:28.925285 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:28.925363 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:28.926026 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:28.926029 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:28.926032 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:28.926042 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:28.926068 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:28.926080 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:28.926164 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:28.926325 (+ 161us) server_negotiation.cc:299] Negotiation successful
0926 11:44:28.926362 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 11:44:28.926652 27366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89158b19e9c642fcab8a1c51b4bc4bc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89158b19e9c642fcab8a1c51b4bc4bc9. Latest consensus state: current_term: 1 leader_uuid: "89158b19e9c642fcab8a1c51b4bc4bc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89158b19e9c642fcab8a1c51b4bc4bc9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42771 } } }
I20260926 11:44:28.926766 27366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89158b19e9c642fcab8a1c51b4bc4bc9 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:28.927073 27365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89158b19e9c642fcab8a1c51b4bc4bc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89158b19e9c642fcab8a1c51b4bc4bc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89158b19e9c642fcab8a1c51b4bc4bc9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42771 } } }
I20260926 11:44:28.927150 27365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89158b19e9c642fcab8a1c51b4bc4bc9 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:28.927546 27374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:28.930570 27374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:28.935536 27374 catalog_manager.cc:1347] Generated new cluster ID: 0e16e84f099944668c354a49f71804ef
I20260926 11:44:28.935604 27374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:28.952539 27374 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:28.952894 27374 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:28.958375 27374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89158b19e9c642fcab8a1c51b4bc4bc9: Generated new TSK 0
I20260926 11:44:28.959139 27374 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:44:28.963198 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 27287
2026-09-26T11:44:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1300 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:44:29 dist-test-slave-xxxx krb5kdc[27386](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:44:29 dist-test-slave-xxxx krb5kdc[27386](info): set up 2 sockets
Sep 26 11:44:29 dist-test-slave-xxxx krb5kdc[27386](info): commencing operation
krb5kdc: starting...
W20260926 11:44:29.914721 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:44:29 dist-test-slave-xxxx krb5kdc[27386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:44:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 11:44:30.027705 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40785
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:30.135581 27399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:30.135852 27399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:30.135895 27399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:30.139842 27399 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:30.139904 27399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:30.139922 27399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:30.139937 27399 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:30.139951 27399 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:30.145234 27399 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:30.146245 27399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:30.147259 27399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:30.155575 27405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:30.155620 27399 server_base.cc:1034] running on GCE node
W20260926 11:44:30.155577 27404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:30.155577 27407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:30.156381 27399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:30.157485 27399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:30.158643 27399 hybrid_clock.cc:648] HybridClock initialized: now 1790423070158623 us; error 33 us; skew 500 ppm
Sep 26 11:44:30 dist-test-slave-xxxx krb5kdc[27386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423070, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:30.165447 27399 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 11:44:30.166323 27399 webserver.cc:460] Webserver started at http://127.0.0.1:36863/ using document root <none> and password file <none>
I20260926 11:44:30.166882 27399 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:30.166944 27399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:30.167121 27399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:30.169086 27399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "f234d97069754cb093fec04af5f1b4ad"
format_stamp: "Formatted at 2026-09-26 11:44:30 on dist-test-slave-xxxx"
server_key: "433c38f28325e4acc7618f0ec48ea4e0"
server_key_iv: "26be6318ecd3e1efa5be828355f67057"
server_key_version: "encryptionkey@0"
I20260926 11:44:30.169634 27399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "f234d97069754cb093fec04af5f1b4ad"
format_stamp: "Formatted at 2026-09-26 11:44:30 on dist-test-slave-xxxx"
server_key: "433c38f28325e4acc7618f0ec48ea4e0"
server_key_iv: "26be6318ecd3e1efa5be828355f67057"
server_key_version: "encryptionkey@0"
I20260926 11:44:30.173108 27399 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 11:44:30.175356 27414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:30.176502 27399 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:44:30.176628 27399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "f234d97069754cb093fec04af5f1b4ad"
format_stamp: "Formatted at 2026-09-26 11:44:30 on dist-test-slave-xxxx"
server_key: "433c38f28325e4acc7618f0ec48ea4e0"
server_key_iv: "26be6318ecd3e1efa5be828355f67057"
server_key_version: "encryptionkey@0"
I20260926 11:44:30.176721 27399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:30.205809 27399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:30.208881 27399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:30.209059 27399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:30.215498 27399 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40785
I20260926 11:44:30.215512 27466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40785 every 8 connection(s)
I20260926 11:44:30.216419 27399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:30.219233 27467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:30.223526 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 27399
I20260926 11:44:30.223647 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:30.223889 6528 external_mini_cluster.cc:1428] Setting key 691612d8a90fce86ed4ba524eea48eca
I20260926 11:44:30.224992 27467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f234d97069754cb093fec04af5f1b4ad: Bootstrap starting.
I20260926 11:44:30.227777 27467 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f234d97069754cb093fec04af5f1b4ad: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:30.228678 27467 log.cc:824] T 00000000000000000000000000000000 P f234d97069754cb093fec04af5f1b4ad: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:30.230664 27467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f234d97069754cb093fec04af5f1b4ad: No bootstrap required, opened a new log
Sep 26 11:44:30 dist-test-slave-xxxx krb5kdc[27386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 11:44:30.234578 27467 raft_consensus.cc:348] T 00000000000000000000000000000000 P f234d97069754cb093fec04af5f1b4ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f234d97069754cb093fec04af5f1b4ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40785 } }
I20260926 11:44:30.234895 27467 raft_consensus.cc:374] T 00000000000000000000000000000000 P f234d97069754cb093fec04af5f1b4ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:30.234953 27467 raft_consensus.cc:729] T 00000000000000000000000000000000 P f234d97069754cb093fec04af5f1b4ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f234d97069754cb093fec04af5f1b4ad, State: Initialized, Role: FOLLOWER
I20260926 11:44:30.235524 27467 consensus_queue.cc:260] T 00000000000000000000000000000000 P f234d97069754cb093fec04af5f1b4ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f234d97069754cb093fec04af5f1b4ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40785 } }
I20260926 11:44:30.235675 27467 raft_consensus.cc:388] T 00000000000000000000000000000000 P f234d97069754cb093fec04af5f1b4ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:44:30.235739 27467 raft_consensus.cc:482] T 00000000000000000000000000000000 P f234d97069754cb093fec04af5f1b4ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:44:30.235836 27467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f234d97069754cb093fec04af5f1b4ad [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:30.236925 27467 raft_consensus.cc:504] T 00000000000000000000000000000000 P f234d97069754cb093fec04af5f1b4ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f234d97069754cb093fec04af5f1b4ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40785 } }
I20260926 11:44:30.237296 27467 leader_election.cc:302] T 00000000000000000000000000000000 P f234d97069754cb093fec04af5f1b4ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f234d97069754cb093fec04af5f1b4ad; no voters:
I20260926 11:44:30.237663 27467 leader_election.cc:288] T 00000000000000000000000000000000 P f234d97069754cb093fec04af5f1b4ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:44:30.237754 27472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f234d97069754cb093fec04af5f1b4ad [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:30.237950 27472 raft_consensus.cc:686] T 00000000000000000000000000000000 P f234d97069754cb093fec04af5f1b4ad [term 1 LEADER]: Becoming Leader. State: Replica: f234d97069754cb093fec04af5f1b4ad, State: Running, Role: LEADER
I20260926 11:44:30.238324 27472 consensus_queue.cc:237] T 00000000000000000000000000000000 P f234d97069754cb093fec04af5f1b4ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f234d97069754cb093fec04af5f1b4ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40785 } }
I20260926 11:44:30.238701 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:30.225176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:42591 (local address 127.0.0.1:40785)
0926 11:44:30.225575 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:30.225596 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:30.225639 (+ 43us) server_negotiation.cc:407] Connection header received
0926 11:44:30.226516 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:30.226541 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:30.227024 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:30.227394 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:30.228092 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:30.229420 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:30.230280 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:30.230994 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:30.231162 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:30.233433 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:30.233439 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:30.233458 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:30.233489 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:30.235352 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:30.235911 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:30.235917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:30.235923 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:30.236065 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:30.236371 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:30.236374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:30.236376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:30.236914 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:30.237148 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:30.237449 (+ 301us) server_negotiation.cc:299] Negotiation successful
0926 11:44:30.237700 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":106,"threads_started":1}
I20260926 11:44:30.239013 27467 sys_catalog.cc:564] T 00000000000000000000000000000000 P f234d97069754cb093fec04af5f1b4ad [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:30.240046 27473 sys_catalog.cc:455] T 00000000000000000000000000000000 P f234d97069754cb093fec04af5f1b4ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f234d97069754cb093fec04af5f1b4ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f234d97069754cb093fec04af5f1b4ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40785 } } }
I20260926 11:44:30.240041 27474 sys_catalog.cc:455] T 00000000000000000000000000000000 P f234d97069754cb093fec04af5f1b4ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader f234d97069754cb093fec04af5f1b4ad. Latest consensus state: current_term: 1 leader_uuid: "f234d97069754cb093fec04af5f1b4ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f234d97069754cb093fec04af5f1b4ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40785 } } }
I20260926 11:44:30.240181 27474 sys_catalog.cc:458] T 00000000000000000000000000000000 P f234d97069754cb093fec04af5f1b4ad [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:30.240181 27473 sys_catalog.cc:458] T 00000000000000000000000000000000 P f234d97069754cb093fec04af5f1b4ad [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:30.240558 27481 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:30.242076 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 11:44:30.244001 27481 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:30.250187 27481 catalog_manager.cc:1347] Generated new cluster ID: 80beb8c5915f43759e4c5508ef61c130
I20260926 11:44:30.250254 27481 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:30.255195 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:30.246341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:37427 (local address 127.0.0.1:40785)
0926 11:44:30.246467 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:30.246471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:30.246747 (+ 276us) server_negotiation.cc:407] Connection header received
0926 11:44:30.246918 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:30.246921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:30.246973 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:30.247066 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:30.248232 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:30.248819 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:30.249607 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:30.249833 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:30.249927 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:30.251592 (+ 1665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:30.251596 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:30.251599 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:30.251646 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:30.253241 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:30.253749 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:30.253753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:30.253754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:30.253841 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:30.254125 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:30.254129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:30.254131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:30.254339 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:30.254486 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:30.254706 (+ 220us) server_negotiation.cc:299] Negotiation successful
0926 11:44:30.255023 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 11:44:30.260993 27481 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:30.261848 27481 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:30.265682 27481 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f234d97069754cb093fec04af5f1b4ad: Generated new TSK 0
I20260926 11:44:30.266294 27481 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:44:30.273808 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 27399
W20260926 11:44:30.278236 27384 connection.cc:535] client connection to 127.0.0.1:40785 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40785): Connection reset by peer (error 104)
2026-09-26T11:44:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1310 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:44:30 dist-test-slave-xxxx krb5kdc[27498](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:44:30 dist-test-slave-xxxx krb5kdc[27498](info): set up 2 sockets
Sep 26 11:44:30 dist-test-slave-xxxx krb5kdc[27498](info): commencing operation
krb5kdc: starting...
W20260926 11:44:31.230916 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:44:31 dist-test-slave-xxxx krb5kdc[27498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:44:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 11:44:31.343426 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36459
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:31.449589 27511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:31.449862 27511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:31.449922 27511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:31.453882 27511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:31.453945 27511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:31.453972 27511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:31.453992 27511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:31.454012 27511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:31.459148 27511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:31.460147 27511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:31.461118 27511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:31.469039 27516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:31.469033 27519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:31.469033 27517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:31.469180 27511 server_base.cc:1034] running on GCE node
I20260926 11:44:31.469890 27511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:31.470942 27511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:31.472096 27511 hybrid_clock.cc:648] HybridClock initialized: now 1790423071472083 us; error 27 us; skew 500 ppm
Sep 26 11:44:31 dist-test-slave-xxxx krb5kdc[27498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423071, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:31.474571 27511 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 11:44:31.475414 27511 webserver.cc:460] Webserver started at http://127.0.0.1:41687/ using document root <none> and password file <none>
I20260926 11:44:31.475898 27511 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:31.475951 27511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:31.476120 27511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:31.478021 27511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "865e7ead99c3448e92c02815700af695"
format_stamp: "Formatted at 2026-09-26 11:44:31 on dist-test-slave-xxxx"
server_key: "aa9b064b97bd759b5537a0c3c8d986cd"
server_key_iv: "68b41a0dca307a203d7c904bc780ce7b"
server_key_version: "encryptionkey@0"
I20260926 11:44:31.478510 27511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "865e7ead99c3448e92c02815700af695"
format_stamp: "Formatted at 2026-09-26 11:44:31 on dist-test-slave-xxxx"
server_key: "aa9b064b97bd759b5537a0c3c8d986cd"
server_key_iv: "68b41a0dca307a203d7c904bc780ce7b"
server_key_version: "encryptionkey@0"
I20260926 11:44:31.481940 27511 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 11:44:31.484176 27526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:31.485316 27511 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 11:44:31.485411 27511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "865e7ead99c3448e92c02815700af695"
format_stamp: "Formatted at 2026-09-26 11:44:31 on dist-test-slave-xxxx"
server_key: "aa9b064b97bd759b5537a0c3c8d986cd"
server_key_iv: "68b41a0dca307a203d7c904bc780ce7b"
server_key_version: "encryptionkey@0"
I20260926 11:44:31.485507 27511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:31.529233 27511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:31.532133 27511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:31.532303 27511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:31.539117 27511 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36459
I20260926 11:44:31.539148 27578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36459 every 8 connection(s)
I20260926 11:44:31.540051 27511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:31.542851 27579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:31.548578 27579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 865e7ead99c3448e92c02815700af695: Bootstrap starting.
I20260926 11:44:31.549309 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 27511
I20260926 11:44:31.549427 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:31.549659 6528 external_mini_cluster.cc:1428] Setting key 80b12c61bd975fb17f1d8ae9e2f3ace7
I20260926 11:44:31.551333 27579 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 865e7ead99c3448e92c02815700af695: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:31.552228 27579 log.cc:824] T 00000000000000000000000000000000 P 865e7ead99c3448e92c02815700af695: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:31.554571 27579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 865e7ead99c3448e92c02815700af695: No bootstrap required, opened a new log
I20260926 11:44:31.558691 27579 raft_consensus.cc:348] T 00000000000000000000000000000000 P 865e7ead99c3448e92c02815700af695 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "865e7ead99c3448e92c02815700af695" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36459 } }
I20260926 11:44:31.558998 27579 raft_consensus.cc:374] T 00000000000000000000000000000000 P 865e7ead99c3448e92c02815700af695 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:31.559108 27579 raft_consensus.cc:729] T 00000000000000000000000000000000 P 865e7ead99c3448e92c02815700af695 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 865e7ead99c3448e92c02815700af695, State: Initialized, Role: FOLLOWER
I20260926 11:44:31.559605 27579 consensus_queue.cc:260] T 00000000000000000000000000000000 P 865e7ead99c3448e92c02815700af695 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "865e7ead99c3448e92c02815700af695" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36459 } }
I20260926 11:44:31.559780 27579 raft_consensus.cc:388] T 00000000000000000000000000000000 P 865e7ead99c3448e92c02815700af695 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:44:31.559844 27579 raft_consensus.cc:482] T 00000000000000000000000000000000 P 865e7ead99c3448e92c02815700af695 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:44:31.559930 27579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 865e7ead99c3448e92c02815700af695 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:31.560380 27582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:31.550920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:55299 (local address 127.0.0.1:36459)
0926 11:44:31.551359 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:31.551380 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:31.551423 (+ 43us) server_negotiation.cc:407] Connection header received
0926 11:44:31.552360 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:31.552385 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:31.552843 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:31.553644 (+ 801us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:31.554059 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:31.555529 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:31.556478 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:31.557236 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:31.557412 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:31.558268 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:31.558274 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:31.558292 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:31.558792 (+ 500us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:31.558887 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:31.558917 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:31.559058 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:31.559252 (+ 194us) server_negotiation.cc:299] Negotiation successful
0926 11:44:31.559384 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":108,"threads_started":1}
I20260926 11:44:31.561075 27579 raft_consensus.cc:504] T 00000000000000000000000000000000 P 865e7ead99c3448e92c02815700af695 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "865e7ead99c3448e92c02815700af695" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36459 } }
I20260926 11:44:31.562232 27579 leader_election.cc:302] T 00000000000000000000000000000000 P 865e7ead99c3448e92c02815700af695 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 865e7ead99c3448e92c02815700af695; no voters:
I20260926 11:44:31.562582 27579 leader_election.cc:288] T 00000000000000000000000000000000 P 865e7ead99c3448e92c02815700af695 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:44:31.562680 27584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 865e7ead99c3448e92c02815700af695 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:31.562940 27584 raft_consensus.cc:686] T 00000000000000000000000000000000 P 865e7ead99c3448e92c02815700af695 [term 1 LEADER]: Becoming Leader. State: Replica: 865e7ead99c3448e92c02815700af695, State: Running, Role: LEADER
I20260926 11:44:31.563285 27584 consensus_queue.cc:237] T 00000000000000000000000000000000 P 865e7ead99c3448e92c02815700af695 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "865e7ead99c3448e92c02815700af695" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36459 } }
I20260926 11:44:31.563426 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 11:44:31.563793 27579 sys_catalog.cc:564] T 00000000000000000000000000000000 P 865e7ead99c3448e92c02815700af695 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:31.564918 27585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 865e7ead99c3448e92c02815700af695 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "865e7ead99c3448e92c02815700af695" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "865e7ead99c3448e92c02815700af695" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36459 } } }
I20260926 11:44:31.565032 27585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 865e7ead99c3448e92c02815700af695 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:31.565240 27585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 865e7ead99c3448e92c02815700af695 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 865e7ead99c3448e92c02815700af695. Latest consensus state: current_term: 1 leader_uuid: "865e7ead99c3448e92c02815700af695" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "865e7ead99c3448e92c02815700af695" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36459 } } }
I20260926 11:44:31.565317 27585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 865e7ead99c3448e92c02815700af695 [sys.catalog]: This master's current role is: LEADER
W20260926 11:44:31.568712 27603 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 865e7ead99c3448e92c02815700af695: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 11:44:31.568779 27603 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 11:44:31.568848 27598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:31.569690 27598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:31.570737 27582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:31.566639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:33299 (local address 127.0.0.1:36459)
0926 11:44:31.566745 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:31.566749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:31.567070 (+ 321us) server_negotiation.cc:407] Connection header received
0926 11:44:31.567156 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:31.567160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:31.567206 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:31.567284 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:31.567486 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:31.568231 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:31.569152 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:31.569393 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:31.569522 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:31.570311 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:31.570315 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:31.570318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:31.570331 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:31.570353 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:31.570363 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:31.570443 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:31.570562 (+ 119us) server_negotiation.cc:299] Negotiation successful
0926 11:44:31.570593 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 11:44:31.575191 27598 catalog_manager.cc:1347] Generated new cluster ID: c506cfd99a6d4ed6af98def53cb08422
I20260926 11:44:31.575253 27598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:31.603950 27598 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:31.605088 27598 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:31.611249 27598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 865e7ead99c3448e92c02815700af695: Generated new TSK 0
I20260926 11:44:31.612007 27598 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:44:31.628010 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 27511
W20260926 11:44:31.632692 27496 connection.cc:535] client connection to 127.0.0.1:36459 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36459): Connection reset by peer (error 104)
2026-09-26T11:44:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1354 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:44:31 dist-test-slave-xxxx krb5kdc[27610](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:44:31 dist-test-slave-xxxx krb5kdc[27610](info): set up 2 sockets
Sep 26 11:44:31 dist-test-slave-xxxx krb5kdc[27610](info): commencing operation
krb5kdc: starting...
W20260926 11:44:32.574733 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.928s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:44:32 dist-test-slave-xxxx krb5kdc[27610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:44:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 11:44:32.689196 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45117
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:32.799297 27623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:32.799587 27623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:32.799647 27623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:32.803679 27623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:32.803745 27623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:32.803771 27623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:32.803791 27623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:32.803817 27623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:32.809113 27623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:32.810194 27623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:32.811230 27623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:32.819658 27628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:32.819733 27623 server_base.cc:1034] running on GCE node
W20260926 11:44:32.819664 27631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:32.819664 27629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:32.820531 27623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:32.821735 27623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:32.822918 27623 hybrid_clock.cc:648] HybridClock initialized: now 1790423072822901 us; error 33 us; skew 500 ppm
Sep 26 11:44:32 dist-test-slave-xxxx krb5kdc[27610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423072, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:32.825625 27623 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 11:44:32.826563 27623 webserver.cc:460] Webserver started at http://127.0.0.1:43411/ using document root <none> and password file <none>
I20260926 11:44:32.827208 27623 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:32.827296 27623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:32.827494 27623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:32.829566 27623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "b6362f1cfe154ef7ae440a03294ded8c"
format_stamp: "Formatted at 2026-09-26 11:44:32 on dist-test-slave-xxxx"
server_key: "75bc1d315d5cc7c3831d867e1694b6e4"
server_key_iv: "832b1d99795766541d1da598f56aaaa5"
server_key_version: "encryptionkey@0"
I20260926 11:44:32.830124 27623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "b6362f1cfe154ef7ae440a03294ded8c"
format_stamp: "Formatted at 2026-09-26 11:44:32 on dist-test-slave-xxxx"
server_key: "75bc1d315d5cc7c3831d867e1694b6e4"
server_key_iv: "832b1d99795766541d1da598f56aaaa5"
server_key_version: "encryptionkey@0"
I20260926 11:44:32.833906 27623 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260926 11:44:32.836374 27638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:32.837605 27623 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 11:44:32.837725 27623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "b6362f1cfe154ef7ae440a03294ded8c"
format_stamp: "Formatted at 2026-09-26 11:44:32 on dist-test-slave-xxxx"
server_key: "75bc1d315d5cc7c3831d867e1694b6e4"
server_key_iv: "832b1d99795766541d1da598f56aaaa5"
server_key_version: "encryptionkey@0"
I20260926 11:44:32.837828 27623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:32.842447 27623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:32.845427 27623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:32.845597 27623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:32.852237 27623 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45117
I20260926 11:44:32.852276 27690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45117 every 8 connection(s)
I20260926 11:44:32.853168 27623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:32.855002 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 27623
I20260926 11:44:32.855139 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:32.855436 6528 external_mini_cluster.cc:1428] Setting key 5f96371b7776ede9a937ac543cbe9cce
I20260926 11:44:32.856220 27691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:32.860625 27694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:32.856875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:44561 (local address 127.0.0.1:45117)
0926 11:44:32.857343 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:32.857357 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:32.857394 (+ 37us) server_negotiation.cc:407] Connection header received
0926 11:44:32.858036 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:32.858052 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:32.858335 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:32.858664 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:32.859465 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:32.859469 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:32.859481 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:32.859499 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:32.859551 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:32.859571 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:32.859697 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:32.859767 (+ 70us) server_negotiation.cc:299] Negotiation successful
0926 11:44:32.859922 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":135,"threads_started":1}
I20260926 11:44:32.862083 27691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6362f1cfe154ef7ae440a03294ded8c: Bootstrap starting.
I20260926 11:44:32.863204 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 11:44:32.864632 27691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6362f1cfe154ef7ae440a03294ded8c: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:32.865487 27691 log.cc:824] T 00000000000000000000000000000000 P b6362f1cfe154ef7ae440a03294ded8c: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:32.867020 27694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:32.865412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:33707 (local address 127.0.0.1:45117)
0926 11:44:32.865563 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:32.865568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:32.865706 (+ 138us) server_negotiation.cc:407] Connection header received
0926 11:44:32.865831 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:32.865834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:32.865871 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:32.865952 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:32.866624 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:32.866628 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:32.866630 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:32.866642 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:32.866668 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:32.866681 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:32.866759 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:32.866840 (+ 81us) server_negotiation.cc:299] Negotiation successful
0926 11:44:32.866883 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 11:44:32.867658 27691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6362f1cfe154ef7ae440a03294ded8c: No bootstrap required, opened a new log
I20260926 11:44:32.871395 27691 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6362f1cfe154ef7ae440a03294ded8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6362f1cfe154ef7ae440a03294ded8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45117 } }
I20260926 11:44:32.871654 27691 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6362f1cfe154ef7ae440a03294ded8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:32.871711 27691 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6362f1cfe154ef7ae440a03294ded8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6362f1cfe154ef7ae440a03294ded8c, State: Initialized, Role: FOLLOWER
I20260926 11:44:32.872113 27691 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6362f1cfe154ef7ae440a03294ded8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6362f1cfe154ef7ae440a03294ded8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45117 } }
I20260926 11:44:32.872237 27691 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6362f1cfe154ef7ae440a03294ded8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:44:32.872301 27691 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6362f1cfe154ef7ae440a03294ded8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:44:32.872398 27691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6362f1cfe154ef7ae440a03294ded8c [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:32.873309 27691 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6362f1cfe154ef7ae440a03294ded8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6362f1cfe154ef7ae440a03294ded8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45117 } }
I20260926 11:44:32.873639 27691 leader_election.cc:302] T 00000000000000000000000000000000 P b6362f1cfe154ef7ae440a03294ded8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6362f1cfe154ef7ae440a03294ded8c; no voters:
I20260926 11:44:32.873952 27691 leader_election.cc:288] T 00000000000000000000000000000000 P b6362f1cfe154ef7ae440a03294ded8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:44:32.874019 27701 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6362f1cfe154ef7ae440a03294ded8c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:32.874234 27701 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6362f1cfe154ef7ae440a03294ded8c [term 1 LEADER]: Becoming Leader. State: Replica: b6362f1cfe154ef7ae440a03294ded8c, State: Running, Role: LEADER
I20260926 11:44:32.874550 27701 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6362f1cfe154ef7ae440a03294ded8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6362f1cfe154ef7ae440a03294ded8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45117 } }
I20260926 11:44:32.875113 27691 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6362f1cfe154ef7ae440a03294ded8c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:32.876221 27703 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6362f1cfe154ef7ae440a03294ded8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6362f1cfe154ef7ae440a03294ded8c. Latest consensus state: current_term: 1 leader_uuid: "b6362f1cfe154ef7ae440a03294ded8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6362f1cfe154ef7ae440a03294ded8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45117 } } }
I20260926 11:44:32.876343 27703 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6362f1cfe154ef7ae440a03294ded8c [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:32.876600 27702 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6362f1cfe154ef7ae440a03294ded8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6362f1cfe154ef7ae440a03294ded8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6362f1cfe154ef7ae440a03294ded8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45117 } } }
I20260926 11:44:32.876693 27702 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6362f1cfe154ef7ae440a03294ded8c [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:32.876782 27710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:32.879570 27710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:32.885308 27710 catalog_manager.cc:1347] Generated new cluster ID: 6184411c3c364d7fa01ad332ecec937c
I20260926 11:44:32.885423 27710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:32.896286 27710 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:32.896613 27710 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:32.901553 27710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6362f1cfe154ef7ae440a03294ded8c: Generated new TSK 0
I20260926 11:44:32.902240 27710 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:44:32.908989 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 27623
2026-09-26T11:44:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1280 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:44:32 dist-test-slave-xxxx krb5kdc[27722](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:44:32 dist-test-slave-xxxx krb5kdc[27722](info): set up 2 sockets
Sep 26 11:44:32 dist-test-slave-xxxx krb5kdc[27722](info): commencing operation
krb5kdc: starting...
W20260926 11:44:33.858796 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.931s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:44:33 dist-test-slave-xxxx krb5kdc[27722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:44:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 11:44:33.977846 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42123
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:34.084352 27735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:34.084609 27735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:34.084666 27735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:34.088615 27735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:34.088672 27735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:34.088697 27735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:34.088717 27735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:34.088739 27735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:34.093885 27735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42123
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:34.094926 27735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:34.095889 27735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:34.103891 27741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:34.103900 27740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:34.103900 27743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:34.104112 27735 server_base.cc:1034] running on GCE node
I20260926 11:44:34.104725 27735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:34.105777 27735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:34.106886 27735 hybrid_clock.cc:648] HybridClock initialized: now 1790423074106872 us; error 30 us; skew 500 ppm
Sep 26 11:44:34 dist-test-slave-xxxx krb5kdc[27722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423074, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:34.109478 27735 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 11:44:34.110316 27735 webserver.cc:460] Webserver started at http://127.0.0.1:43133/ using document root <none> and password file <none>
I20260926 11:44:34.110879 27735 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:34.110967 27735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:34.111172 27735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:34.113175 27735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "fc41dde55e7b442daeb5c4148ae5384b"
format_stamp: "Formatted at 2026-09-26 11:44:34 on dist-test-slave-xxxx"
server_key: "afb2026bad1b6b2934d2adf00a691cd6"
server_key_iv: "fa5de7d93c1fc68f667cb75b5027167d"
server_key_version: "encryptionkey@0"
I20260926 11:44:34.113709 27735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "fc41dde55e7b442daeb5c4148ae5384b"
format_stamp: "Formatted at 2026-09-26 11:44:34 on dist-test-slave-xxxx"
server_key: "afb2026bad1b6b2934d2adf00a691cd6"
server_key_iv: "fa5de7d93c1fc68f667cb75b5027167d"
server_key_version: "encryptionkey@0"
I20260926 11:44:34.117389 27735 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260926 11:44:34.119799 27750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:34.121060 27735 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:44:34.121178 27735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "fc41dde55e7b442daeb5c4148ae5384b"
format_stamp: "Formatted at 2026-09-26 11:44:34 on dist-test-slave-xxxx"
server_key: "afb2026bad1b6b2934d2adf00a691cd6"
server_key_iv: "fa5de7d93c1fc68f667cb75b5027167d"
server_key_version: "encryptionkey@0"
I20260926 11:44:34.121281 27735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:34.138743 27735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:34.141467 27735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:34.141638 27735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:34.148319 27735 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42123
I20260926 11:44:34.148336 27802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42123 every 8 connection(s)
I20260926 11:44:34.149224 27735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:34.151876 27803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:34.153389 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 27735
I20260926 11:44:34.153535 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:34.153836 6528 external_mini_cluster.cc:1428] Setting key 85982841873141031ef887da204336fc
I20260926 11:44:34.158401 27803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc41dde55e7b442daeb5c4148ae5384b: Bootstrap starting.
I20260926 11:44:34.160982 27803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc41dde55e7b442daeb5c4148ae5384b: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:34.161718 27803 log.cc:824] T 00000000000000000000000000000000 P fc41dde55e7b442daeb5c4148ae5384b: Log is configured to *not* fsync() on all Append() calls
Sep 26 11:44:34 dist-test-slave-xxxx krb5kdc[27722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 11:44:34.163911 27803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc41dde55e7b442daeb5c4148ae5384b: No bootstrap required, opened a new log
I20260926 11:44:34.167797 27803 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc41dde55e7b442daeb5c4148ae5384b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc41dde55e7b442daeb5c4148ae5384b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42123 } }
I20260926 11:44:34.168016 27803 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc41dde55e7b442daeb5c4148ae5384b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:34.168059 27803 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc41dde55e7b442daeb5c4148ae5384b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc41dde55e7b442daeb5c4148ae5384b, State: Initialized, Role: FOLLOWER
I20260926 11:44:34.168159 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:34.155184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:56365 (local address 127.0.0.1:42123)
0926 11:44:34.155585 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:34.155605 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:34.155646 (+ 41us) server_negotiation.cc:407] Connection header received
0926 11:44:34.156442 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:34.156467 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:34.156893 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:34.157258 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:34.157946 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:34.159371 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:34.160204 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:34.160890 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:34.161044 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:34.163762 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:34.163767 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:34.163780 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:34.163805 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:34.165320 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:34.165882 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:34.165886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:34.165891 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:34.165998 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:34.166304 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:34.166307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:34.166308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:34.166690 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:34.166893 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:34.167162 (+ 269us) server_negotiation.cc:299] Negotiation successful
0926 11:44:34.167367 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":99,"threads_started":1}
I20260926 11:44:34.168524 27803 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc41dde55e7b442daeb5c4148ae5384b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc41dde55e7b442daeb5c4148ae5384b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42123 } }
I20260926 11:44:34.168654 27803 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc41dde55e7b442daeb5c4148ae5384b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:44:34.168716 27803 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc41dde55e7b442daeb5c4148ae5384b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:44:34.168810 27803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc41dde55e7b442daeb5c4148ae5384b [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:34.169821 27803 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc41dde55e7b442daeb5c4148ae5384b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc41dde55e7b442daeb5c4148ae5384b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42123 } }
I20260926 11:44:34.170156 27803 leader_election.cc:302] T 00000000000000000000000000000000 P fc41dde55e7b442daeb5c4148ae5384b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc41dde55e7b442daeb5c4148ae5384b; no voters:
I20260926 11:44:34.170477 27803 leader_election.cc:288] T 00000000000000000000000000000000 P fc41dde55e7b442daeb5c4148ae5384b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:44:34.170580 27808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc41dde55e7b442daeb5c4148ae5384b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:34.170781 27808 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc41dde55e7b442daeb5c4148ae5384b [term 1 LEADER]: Becoming Leader. State: Replica: fc41dde55e7b442daeb5c4148ae5384b, State: Running, Role: LEADER
I20260926 11:44:34.171113 27808 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc41dde55e7b442daeb5c4148ae5384b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc41dde55e7b442daeb5c4148ae5384b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42123 } }
I20260926 11:44:34.171175 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 11:44:34.171731 27803 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc41dde55e7b442daeb5c4148ae5384b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:34.172801 27810 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc41dde55e7b442daeb5c4148ae5384b [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc41dde55e7b442daeb5c4148ae5384b. Latest consensus state: current_term: 1 leader_uuid: "fc41dde55e7b442daeb5c4148ae5384b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc41dde55e7b442daeb5c4148ae5384b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42123 } } }
I20260926 11:44:34.172923 27810 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc41dde55e7b442daeb5c4148ae5384b [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:34.173105 27809 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc41dde55e7b442daeb5c4148ae5384b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc41dde55e7b442daeb5c4148ae5384b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc41dde55e7b442daeb5c4148ae5384b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42123 } } }
I20260926 11:44:34.173185 27809 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc41dde55e7b442daeb5c4148ae5384b [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:34.173260 27821 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:34.176409 27821 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:34.181345 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:34.173247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:43131 (local address 127.0.0.1:42123)
0926 11:44:34.173336 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:34.173340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:34.173802 (+ 462us) server_negotiation.cc:407] Connection header received
0926 11:44:34.173921 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:34.173925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:34.173973 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:34.174046 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:34.174884 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:34.175606 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:34.176706 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:34.177018 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:34.177105 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:34.178371 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:34.178375 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:34.178378 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:34.178401 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:34.179657 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:34.180081 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:34.180084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:34.180086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:34.180163 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:34.180440 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:34.180443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:34.180445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:34.180639 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:34.180769 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:34.181116 (+ 347us) server_negotiation.cc:299] Negotiation successful
0926 11:44:34.181209 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 11:44:34.182677 27821 catalog_manager.cc:1347] Generated new cluster ID: 1414e6b79d624d6fb491aa0cae616d10
I20260926 11:44:34.182723 27821 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:34.196177 27821 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:34.197091 27821 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:34.200747 27821 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc41dde55e7b442daeb5c4148ae5384b: Generated new TSK 0
I20260926 11:44:34.201530 27821 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:44:34.205617 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 27735
W20260926 11:44:34.210179 27720 connection.cc:535] client connection to 127.0.0.1:42123 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42123): Connection reset by peer (error 104)
2026-09-26T11:44:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1296 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:44:34 dist-test-slave-xxxx krb5kdc[27834](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:44:34 dist-test-slave-xxxx krb5kdc[27834](info): set up 2 sockets
Sep 26 11:44:34 dist-test-slave-xxxx krb5kdc[27834](info): commencing operation
krb5kdc: starting...
W20260926 11:44:35.158879 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:44:35 dist-test-slave-xxxx krb5kdc[27834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:44:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 11:44:35.276046 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32853
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:35.382385 27847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:35.382649 27847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:35.382689 27847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:35.386627 27847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:35.386694 27847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:35.386711 27847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:35.386727 27847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:35.386740 27847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:35.392064 27847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32853
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:35.393138 27847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:35.394151 27847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:35.403913 27852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:35.404003 27853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:35.404182 27847 server_base.cc:1034] running on GCE node
W20260926 11:44:35.404052 27855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:35.404978 27847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:35.406141 27847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:35.407310 27847 hybrid_clock.cc:648] HybridClock initialized: now 1790423075407292 us; error 38 us; skew 500 ppm
Sep 26 11:44:35 dist-test-slave-xxxx krb5kdc[27834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423075, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:35.414690 27847 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 11:44:35.415710 27847 webserver.cc:460] Webserver started at http://127.0.0.1:44253/ using document root <none> and password file <none>
I20260926 11:44:35.416263 27847 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:35.416332 27847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:35.416553 27847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:35.418613 27847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "ff39f4ca7bce4a8ab421293395ba47d3"
format_stamp: "Formatted at 2026-09-26 11:44:35 on dist-test-slave-xxxx"
server_key: "ba787c95a61b84dfcdf043b821d1314d"
server_key_iv: "336a32ef871c4251ac091155cc5c2c54"
server_key_version: "encryptionkey@0"
I20260926 11:44:35.419200 27847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "ff39f4ca7bce4a8ab421293395ba47d3"
format_stamp: "Formatted at 2026-09-26 11:44:35 on dist-test-slave-xxxx"
server_key: "ba787c95a61b84dfcdf043b821d1314d"
server_key_iv: "336a32ef871c4251ac091155cc5c2c54"
server_key_version: "encryptionkey@0"
I20260926 11:44:35.423071 27847 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 11:44:35.425529 27862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:35.426776 27847 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:44:35.426916 27847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "ff39f4ca7bce4a8ab421293395ba47d3"
format_stamp: "Formatted at 2026-09-26 11:44:35 on dist-test-slave-xxxx"
server_key: "ba787c95a61b84dfcdf043b821d1314d"
server_key_iv: "336a32ef871c4251ac091155cc5c2c54"
server_key_version: "encryptionkey@0"
I20260926 11:44:35.427013 27847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:35.449679 27847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:35.452970 27847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:35.453156 27847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:35.459949 27847 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32853
I20260926 11:44:35.459956 27914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32853 every 8 connection(s)
I20260926 11:44:35.460886 27847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:35.462185 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 27847
I20260926 11:44:35.462318 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:35.462610 6528 external_mini_cluster.cc:1428] Setting key 905256bf8c31aef5e7da69920bfb1b67
I20260926 11:44:35.464109 27915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:35.470256 27915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff39f4ca7bce4a8ab421293395ba47d3: Bootstrap starting.
I20260926 11:44:35.471505 27918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:35.463984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:34473 (local address 127.0.0.1:32853)
0926 11:44:35.464419 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:35.464453 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:35.464489 (+ 36us) server_negotiation.cc:407] Connection header received
0926 11:44:35.465113 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:35.465128 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:35.465471 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:35.465757 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:35.466102 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:35.467148 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:35.468347 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:35.468831 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:35.468948 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:35.469851 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:35.469856 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:35.469868 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:35.470262 (+ 394us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:35.470316 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:35.470337 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:35.470481 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:35.470618 (+ 137us) server_negotiation.cc:299] Negotiation successful
0926 11:44:35.470750 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":144,"threads_started":1}
I20260926 11:44:35.472659 27915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff39f4ca7bce4a8ab421293395ba47d3: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:35.473484 27915 log.cc:824] T 00000000000000000000000000000000 P ff39f4ca7bce4a8ab421293395ba47d3: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:35.474378 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 11:44:35.475591 27915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff39f4ca7bce4a8ab421293395ba47d3: No bootstrap required, opened a new log
I20260926 11:44:35.479341 27915 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff39f4ca7bce4a8ab421293395ba47d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff39f4ca7bce4a8ab421293395ba47d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32853 } }
I20260926 11:44:35.479637 27915 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff39f4ca7bce4a8ab421293395ba47d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:35.479699 27915 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff39f4ca7bce4a8ab421293395ba47d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff39f4ca7bce4a8ab421293395ba47d3, State: Initialized, Role: FOLLOWER
I20260926 11:44:35.480134 27915 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff39f4ca7bce4a8ab421293395ba47d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff39f4ca7bce4a8ab421293395ba47d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32853 } }
I20260926 11:44:35.480261 27915 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff39f4ca7bce4a8ab421293395ba47d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:44:35.480324 27915 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff39f4ca7bce4a8ab421293395ba47d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:44:35.480422 27915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff39f4ca7bce4a8ab421293395ba47d3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:35.480877 27918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:35.476519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:40173 (local address 127.0.0.1:32853)
0926 11:44:35.476630 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:35.476635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:35.476833 (+ 198us) server_negotiation.cc:407] Connection header received
0926 11:44:35.476913 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:35.476916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:35.476962 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:35.477042 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:35.477240 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:35.478433 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:35.479245 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:35.479575 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:35.479684 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:35.480460 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:35.480463 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:35.480466 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:35.480481 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:35.480511 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:35.480525 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:35.480616 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:35.480719 (+ 103us) server_negotiation.cc:299] Negotiation successful
0926 11:44:35.480754 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 11:44:35.481511 27915 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff39f4ca7bce4a8ab421293395ba47d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff39f4ca7bce4a8ab421293395ba47d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32853 } }
I20260926 11:44:35.481864 27915 leader_election.cc:302] T 00000000000000000000000000000000 P ff39f4ca7bce4a8ab421293395ba47d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff39f4ca7bce4a8ab421293395ba47d3; no voters:
I20260926 11:44:35.482163 27915 leader_election.cc:288] T 00000000000000000000000000000000 P ff39f4ca7bce4a8ab421293395ba47d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:44:35.482256 27925 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff39f4ca7bce4a8ab421293395ba47d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:35.482462 27925 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff39f4ca7bce4a8ab421293395ba47d3 [term 1 LEADER]: Becoming Leader. State: Replica: ff39f4ca7bce4a8ab421293395ba47d3, State: Running, Role: LEADER
I20260926 11:44:35.482746 27925 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff39f4ca7bce4a8ab421293395ba47d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff39f4ca7bce4a8ab421293395ba47d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32853 } }
I20260926 11:44:35.483314 27915 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff39f4ca7bce4a8ab421293395ba47d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:35.484251 27926 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff39f4ca7bce4a8ab421293395ba47d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff39f4ca7bce4a8ab421293395ba47d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff39f4ca7bce4a8ab421293395ba47d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32853 } } }
I20260926 11:44:35.484371 27926 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff39f4ca7bce4a8ab421293395ba47d3 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:35.484683 27927 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff39f4ca7bce4a8ab421293395ba47d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff39f4ca7bce4a8ab421293395ba47d3. Latest consensus state: current_term: 1 leader_uuid: "ff39f4ca7bce4a8ab421293395ba47d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff39f4ca7bce4a8ab421293395ba47d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32853 } } }
I20260926 11:44:35.484767 27927 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff39f4ca7bce4a8ab421293395ba47d3 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:35.484884 27934 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:35.487895 27934 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:35.493196 27934 catalog_manager.cc:1347] Generated new cluster ID: f97f85485a8541e59862edb32a280723
I20260926 11:44:35.493265 27934 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:35.513375 27934 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:35.514693 27934 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:35.527388 27934 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff39f4ca7bce4a8ab421293395ba47d3: Generated new TSK 0
I20260926 11:44:35.528271 27934 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:44:35.545145 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 27847
W20260926 11:44:35.549804 27832 connection.cc:535] client connection to 127.0.0.1:32853 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32853): Connection reset by peer (error 104)
2026-09-26T11:44:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1339 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:44:35 dist-test-slave-xxxx krb5kdc[27946](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:44:35 dist-test-slave-xxxx krb5kdc[27946](info): set up 2 sockets
Sep 26 11:44:35 dist-test-slave-xxxx krb5kdc[27946](info): commencing operation
krb5kdc: starting...
W20260926 11:44:36.498669 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:44:36 dist-test-slave-xxxx krb5kdc[27946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:44:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 11:44:36.617432 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37459
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:36.726091 27959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:36.726401 27959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:36.726444 27959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:36.730508 27959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:36.730571 27959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:36.730590 27959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:36.730605 27959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:36.730618 27959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:36.735965 27959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37459
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:36.736979 27959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:36.737968 27959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:36.746248 27967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:36.746227 27965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:36.746230 27964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:36.746292 27959 server_base.cc:1034] running on GCE node
I20260926 11:44:36.747085 27959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:36.748159 27959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:36.749274 27959 hybrid_clock.cc:648] HybridClock initialized: now 1790423076749245 us; error 39 us; skew 500 ppm
Sep 26 11:44:36 dist-test-slave-xxxx krb5kdc[27946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423076, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:36.751988 27959 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 11:44:36.752839 27959 webserver.cc:460] Webserver started at http://127.0.0.1:33691/ using document root <none> and password file <none>
I20260926 11:44:36.753364 27959 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:36.753427 27959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:36.753612 27959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:36.755642 27959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "c99a14c30593413995525ccfae86dfd9"
format_stamp: "Formatted at 2026-09-26 11:44:36 on dist-test-slave-xxxx"
server_key: "52748c38cfbb88515d1227bd3423bdcf"
server_key_iv: "8e07ce547262dd4eba0628776e6557b5"
server_key_version: "encryptionkey@0"
I20260926 11:44:36.756191 27959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "c99a14c30593413995525ccfae86dfd9"
format_stamp: "Formatted at 2026-09-26 11:44:36 on dist-test-slave-xxxx"
server_key: "52748c38cfbb88515d1227bd3423bdcf"
server_key_iv: "8e07ce547262dd4eba0628776e6557b5"
server_key_version: "encryptionkey@0"
I20260926 11:44:36.760046 27959 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 11:44:36.762318 27974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:36.763594 27959 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:44:36.763710 27959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "c99a14c30593413995525ccfae86dfd9"
format_stamp: "Formatted at 2026-09-26 11:44:36 on dist-test-slave-xxxx"
server_key: "52748c38cfbb88515d1227bd3423bdcf"
server_key_iv: "8e07ce547262dd4eba0628776e6557b5"
server_key_version: "encryptionkey@0"
I20260926 11:44:36.763809 27959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:36.768211 27959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:36.771179 27959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:36.771340 27959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:36.777724 27959 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37459
I20260926 11:44:36.777765 28026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37459 every 8 connection(s)
I20260926 11:44:36.778606 27959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:36.781425 28027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:36.783094 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 27959
I20260926 11:44:36.783226 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:36.783470 6528 external_mini_cluster.cc:1428] Setting key 785ea612e591a27b77380d971e0997e5
I20260926 11:44:36.787684 28027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c99a14c30593413995525ccfae86dfd9: Bootstrap starting.
I20260926 11:44:36.788651 28030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:36.784864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:46499 (local address 127.0.0.1:37459)
0926 11:44:36.785332 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:36.785348 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:36.785384 (+ 36us) server_negotiation.cc:407] Connection header received
0926 11:44:36.786021 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:36.786036 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:36.786336 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:36.786640 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:36.787426 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:36.787432 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:36.787445 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:36.787455 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:36.787506 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:36.787526 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:36.787646 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:36.787715 (+ 69us) server_negotiation.cc:299] Negotiation successful
0926 11:44:36.787973 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":133,"threads_started":1}
I20260926 11:44:36.789958 28027 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c99a14c30593413995525ccfae86dfd9: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:36.790764 28027 log.cc:824] T 00000000000000000000000000000000 P c99a14c30593413995525ccfae86dfd9: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:36.791544 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 11:44:36.792691 28027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c99a14c30593413995525ccfae86dfd9: No bootstrap required, opened a new log
I20260926 11:44:36.794800 28030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:36.793392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.56:50929 (local address 127.0.0.1:37459)
0926 11:44:36.793502 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:36.793506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:36.793633 (+ 127us) server_negotiation.cc:407] Connection header received
0926 11:44:36.793715 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:36.793718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:36.793752 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:36.793823 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:36.794461 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:36.794465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:36.794468 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:36.794478 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:36.794504 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:36.794515 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:36.794596 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:36.794646 (+ 50us) server_negotiation.cc:299] Negotiation successful
0926 11:44:36.794683 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 11:44:36.796183 28027 raft_consensus.cc:348] T 00000000000000000000000000000000 P c99a14c30593413995525ccfae86dfd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c99a14c30593413995525ccfae86dfd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37459 } }
I20260926 11:44:36.796422 28027 raft_consensus.cc:374] T 00000000000000000000000000000000 P c99a14c30593413995525ccfae86dfd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:36.796461 28027 raft_consensus.cc:729] T 00000000000000000000000000000000 P c99a14c30593413995525ccfae86dfd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c99a14c30593413995525ccfae86dfd9, State: Initialized, Role: FOLLOWER
I20260926 11:44:36.796824 28027 consensus_queue.cc:260] T 00000000000000000000000000000000 P c99a14c30593413995525ccfae86dfd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c99a14c30593413995525ccfae86dfd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37459 } }
I20260926 11:44:36.796934 28027 raft_consensus.cc:388] T 00000000000000000000000000000000 P c99a14c30593413995525ccfae86dfd9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:44:36.796975 28027 raft_consensus.cc:482] T 00000000000000000000000000000000 P c99a14c30593413995525ccfae86dfd9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:44:36.797043 28027 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c99a14c30593413995525ccfae86dfd9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:36.797870 28027 raft_consensus.cc:504] T 00000000000000000000000000000000 P c99a14c30593413995525ccfae86dfd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c99a14c30593413995525ccfae86dfd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37459 } }
I20260926 11:44:36.798161 28027 leader_election.cc:302] T 00000000000000000000000000000000 P c99a14c30593413995525ccfae86dfd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c99a14c30593413995525ccfae86dfd9; no voters:
I20260926 11:44:36.798412 28027 leader_election.cc:288] T 00000000000000000000000000000000 P c99a14c30593413995525ccfae86dfd9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:44:36.798516 28037 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c99a14c30593413995525ccfae86dfd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:36.798712 28037 raft_consensus.cc:686] T 00000000000000000000000000000000 P c99a14c30593413995525ccfae86dfd9 [term 1 LEADER]: Becoming Leader. State: Replica: c99a14c30593413995525ccfae86dfd9, State: Running, Role: LEADER
I20260926 11:44:36.799016 28037 consensus_queue.cc:237] T 00000000000000000000000000000000 P c99a14c30593413995525ccfae86dfd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c99a14c30593413995525ccfae86dfd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37459 } }
I20260926 11:44:36.799583 28027 sys_catalog.cc:564] T 00000000000000000000000000000000 P c99a14c30593413995525ccfae86dfd9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:36.800621 28038 sys_catalog.cc:455] T 00000000000000000000000000000000 P c99a14c30593413995525ccfae86dfd9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c99a14c30593413995525ccfae86dfd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c99a14c30593413995525ccfae86dfd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37459 } } }
I20260926 11:44:36.800740 28038 sys_catalog.cc:458] T 00000000000000000000000000000000 P c99a14c30593413995525ccfae86dfd9 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:36.800978 28039 sys_catalog.cc:455] T 00000000000000000000000000000000 P c99a14c30593413995525ccfae86dfd9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c99a14c30593413995525ccfae86dfd9. Latest consensus state: current_term: 1 leader_uuid: "c99a14c30593413995525ccfae86dfd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c99a14c30593413995525ccfae86dfd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37459 } } }
I20260926 11:44:36.801054 28039 sys_catalog.cc:458] T 00000000000000000000000000000000 P c99a14c30593413995525ccfae86dfd9 [sys.catalog]: This master's current role is: LEADER
W20260926 11:44:36.804229 28052 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c99a14c30593413995525ccfae86dfd9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 11:44:36.804307 28052 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 11:44:36.804376 28046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:36.805074 28046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:36.810391 28046 catalog_manager.cc:1347] Generated new cluster ID: 2839c16073224afea807d6f4c71c5197
I20260926 11:44:36.810451 28046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:36.833933 28046 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:36.834331 28046 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:36.844501 28046 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c99a14c30593413995525ccfae86dfd9: Generated new TSK 0
I20260926 11:44:36.845243 28046 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:44:36.860847 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 27959
2026-09-26T11:44:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1315 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:44:36 dist-test-slave-xxxx krb5kdc[28058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:44:36 dist-test-slave-xxxx krb5kdc[28058](info): set up 2 sockets
Sep 26 11:44:36 dist-test-slave-xxxx krb5kdc[28058](info): commencing operation
krb5kdc: starting...
W20260926 11:44:37.822788 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:44:37 dist-test-slave-xxxx krb5kdc[28058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:44:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:44:37.938649 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:40155
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:32969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:40155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:38.046129 28071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:38.046409 28071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:38.046452 28071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:38.050457 28071 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:38.050514 28071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:38.050532 28071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:38.050547 28071 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:38.050562 28071 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:38.055755 28071 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:32969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:40155
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:40155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:38.056747 28071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:38.057751 28071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:38.065701 28079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:38.065786 28071 server_base.cc:1034] running on GCE node
W20260926 11:44:38.065835 28077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:38.065709 28076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:38.066623 28071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:38.067718 28071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:38.068902 28071 hybrid_clock.cc:648] HybridClock initialized: now 1790423078068883 us; error 27 us; skew 500 ppm
Sep 26 11:44:38 dist-test-slave-xxxx krb5kdc[28058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423078, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:38.071462 28071 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:44:38.072256 28071 webserver.cc:460] Webserver started at http://127.6.96.62:38911/ using document root <none> and password file <none>
I20260926 11:44:38.072759 28071 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:38.072815 28071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:38.072969 28071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:38.074940 28071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "e6892174cf584132b4020eea5c4710f0"
format_stamp: "Formatted at 2026-09-26 11:44:38 on dist-test-slave-xxxx"
server_key: "1d93c1eab5b347ecb346bf2e3b1f8052"
server_key_iv: "7cb4f4e89850922fe43367de8d0d12b1"
server_key_version: "encryptionkey@0"
I20260926 11:44:38.075452 28071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "e6892174cf584132b4020eea5c4710f0"
format_stamp: "Formatted at 2026-09-26 11:44:38 on dist-test-slave-xxxx"
server_key: "1d93c1eab5b347ecb346bf2e3b1f8052"
server_key_iv: "7cb4f4e89850922fe43367de8d0d12b1"
server_key_version: "encryptionkey@0"
I20260926 11:44:38.079068 28071 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:44:38.081460 28086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:38.082690 28071 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:44:38.082800 28071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "e6892174cf584132b4020eea5c4710f0"
format_stamp: "Formatted at 2026-09-26 11:44:38 on dist-test-slave-xxxx"
server_key: "1d93c1eab5b347ecb346bf2e3b1f8052"
server_key_iv: "7cb4f4e89850922fe43367de8d0d12b1"
server_key_version: "encryptionkey@0"
I20260926 11:44:38.082962 28071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:38.094583 28071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:38.097374 28071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:38.097529 28071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:38.104269 28071 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:40155
I20260926 11:44:38.104322 28138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:40155 every 8 connection(s)
I20260926 11:44:38.105265 28071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:38.108040 28139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:38.113588 28139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6892174cf584132b4020eea5c4710f0: Bootstrap starting.
I20260926 11:44:38.114360 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 28071
I20260926 11:44:38.114503 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:38.114771 6528 external_mini_cluster.cc:1428] Setting key 37b9ebc09f996dc6996c95041135aa78
I20260926 11:44:38.116655 28139 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6892174cf584132b4020eea5c4710f0: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:38.117396 28139 log.cc:824] T 00000000000000000000000000000000 P e6892174cf584132b4020eea5c4710f0: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:38.119640 28139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6892174cf584132b4020eea5c4710f0: No bootstrap required, opened a new log
Sep 26 11:44:38 dist-test-slave-xxxx krb5kdc[28058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:44:38.123890 28139 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6892174cf584132b4020eea5c4710f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6892174cf584132b4020eea5c4710f0" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40155 } }
I20260926 11:44:38.124146 28139 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6892174cf584132b4020eea5c4710f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:38.124207 28139 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6892174cf584132b4020eea5c4710f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6892174cf584132b4020eea5c4710f0, State: Initialized, Role: FOLLOWER
I20260926 11:44:38.124687 28139 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6892174cf584132b4020eea5c4710f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6892174cf584132b4020eea5c4710f0" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40155 } }
I20260926 11:44:38.124836 28139 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6892174cf584132b4020eea5c4710f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:44:38.124900 28139 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6892174cf584132b4020eea5c4710f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:44:38.125000 28139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6892174cf584132b4020eea5c4710f0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:38.125873 28139 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6892174cf584132b4020eea5c4710f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6892174cf584132b4020eea5c4710f0" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40155 } }
I20260926 11:44:38.126223 28139 leader_election.cc:302] T 00000000000000000000000000000000 P e6892174cf584132b4020eea5c4710f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6892174cf584132b4020eea5c4710f0; no voters:
I20260926 11:44:38.126541 28139 leader_election.cc:288] T 00000000000000000000000000000000 P e6892174cf584132b4020eea5c4710f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:44:38.126629 28144 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6892174cf584132b4020eea5c4710f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:38.126849 28144 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6892174cf584132b4020eea5c4710f0 [term 1 LEADER]: Becoming Leader. State: Replica: e6892174cf584132b4020eea5c4710f0, State: Running, Role: LEADER
I20260926 11:44:38.127206 28144 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6892174cf584132b4020eea5c4710f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6892174cf584132b4020eea5c4710f0" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40155 } }
I20260926 11:44:38.127929 28139 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6892174cf584132b4020eea5c4710f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:38.128394 28144 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6892174cf584132b4020eea5c4710f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6892174cf584132b4020eea5c4710f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6892174cf584132b4020eea5c4710f0" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40155 } } }
I20260926 11:44:38.128566 28144 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6892174cf584132b4020eea5c4710f0 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:38.128794 28145 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6892174cf584132b4020eea5c4710f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6892174cf584132b4020eea5c4710f0. Latest consensus state: current_term: 1 leader_uuid: "e6892174cf584132b4020eea5c4710f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6892174cf584132b4020eea5c4710f0" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40155 } } }
I20260926 11:44:38.128954 28145 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6892174cf584132b4020eea5c4710f0 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:38.128989 28149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:38.129736 28142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:38.116114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57541 (local address 127.6.96.62:40155)
0926 11:44:38.116534 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:38.116550 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:38.116588 (+ 38us) server_negotiation.cc:407] Connection header received
0926 11:44:38.117311 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:38.117330 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:38.117718 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:38.118113 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:38.118974 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:38.120031 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:38.121177 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:38.121641 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:38.121760 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:38.124582 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:38.124587 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:38.124601 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:38.124628 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:38.126447 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:38.127051 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:38.127057 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:38.127063 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:38.127226 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:38.127585 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:38.127588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:38.127590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:38.128027 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:38.128233 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:38.128590 (+ 357us) server_negotiation.cc:299] Negotiation successful
0926 11:44:38.128839 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":118,"threads_started":1}
I20260926 11:44:38.132304 28149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:38.133307 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 11:44:38.141793 28149 catalog_manager.cc:1347] Generated new cluster ID: a56c3c1f83a84b0faa1d20dbbe01bc3b
I20260926 11:44:38.141957 28149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:38.143831 28142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:38.135576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48323 (local address 127.6.96.62:40155)
0926 11:44:38.135681 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:38.135685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:38.135935 (+ 250us) server_negotiation.cc:407] Connection header received
0926 11:44:38.136132 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:38.136135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:38.136181 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:38.136230 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:38.137117 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:38.137780 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:38.138619 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:38.138886 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:38.138973 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:38.140459 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:38.140462 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:38.140465 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:38.140491 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:38.142183 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:38.142613 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:38.142617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:38.142619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:38.142690 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:38.143024 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:38.143027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:38.143029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:38.143193 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:38.143316 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:38.143574 (+ 258us) server_negotiation.cc:299] Negotiation successful
0926 11:44:38.143660 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":39}
I20260926 11:44:38.155375 28149 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:38.156344 28149 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:38.169662 28149 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6892174cf584132b4020eea5c4710f0: Generated new TSK 0
I20260926 11:44:38.170554 28149 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:44:38.182191 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 28071
W20260926 11:44:38.186686 28056 connection.cc:535] client connection to 127.6.96.62:40155 recv error: Network error: failed to read from TLS socket (remote: 127.6.96.62:40155): Connection reset by peer (error 104)
2026-09-26T11:44:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1321 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:44:38 dist-test-slave-xxxx krb5kdc[28170](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:44:38 dist-test-slave-xxxx krb5kdc[28170](info): set up 2 sockets
Sep 26 11:44:38 dist-test-slave-xxxx krb5kdc[28170](info): commencing operation
krb5kdc: starting...
W20260926 11:44:39.134716 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:44:39 dist-test-slave-xxxx krb5kdc[28170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423079, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:44:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:44:39.249749 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:45763
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:45763
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:39.356316 28183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:39.356580 28183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:39.356637 28183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:39.360582 28183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:39.360642 28183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:39.360668 28183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:39.360690 28183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:39.360715 28183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:39.365859 28183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:45763
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:45763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:39.366896 28183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:39.367882 28183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 11:44:39.376067 28183 server_base.cc:1034] running on GCE node
W20260926 11:44:39.376024 28188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:39.376057 28189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:39.376029 28191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:39.376775 28183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:39.377844 28183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:39.378993 28183 hybrid_clock.cc:648] HybridClock initialized: now 1790423079378979 us; error 28 us; skew 500 ppm
Sep 26 11:44:39 dist-test-slave-xxxx krb5kdc[28170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423079, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:39.381584 28183 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:44:39.382417 28183 webserver.cc:460] Webserver started at http://127.6.96.62:43407/ using document root <none> and password file <none>
I20260926 11:44:39.382961 28183 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:39.383023 28183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:39.383185 28183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:39.385174 28183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "d8637d5a710e46ae979d69f9d276e3d9"
format_stamp: "Formatted at 2026-09-26 11:44:39 on dist-test-slave-xxxx"
server_key: "5614e9bdaad183fa49994ef3a7adcc0f"
server_key_iv: "83c4e6df7db4d931a9b628472d03894e"
server_key_version: "encryptionkey@0"
I20260926 11:44:39.385670 28183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "d8637d5a710e46ae979d69f9d276e3d9"
format_stamp: "Formatted at 2026-09-26 11:44:39 on dist-test-slave-xxxx"
server_key: "5614e9bdaad183fa49994ef3a7adcc0f"
server_key_iv: "83c4e6df7db4d931a9b628472d03894e"
server_key_version: "encryptionkey@0"
I20260926 11:44:39.389207 28183 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 11:44:39.391500 28198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:39.392649 28183 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:44:39.392748 28183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "d8637d5a710e46ae979d69f9d276e3d9"
format_stamp: "Formatted at 2026-09-26 11:44:39 on dist-test-slave-xxxx"
server_key: "5614e9bdaad183fa49994ef3a7adcc0f"
server_key_iv: "83c4e6df7db4d931a9b628472d03894e"
server_key_version: "encryptionkey@0"
I20260926 11:44:39.392858 28183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:39.415448 28183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:39.418221 28183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:39.418390 28183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:39.424970 28183 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:45763
I20260926 11:44:39.425020 28250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:45763 every 8 connection(s)
I20260926 11:44:39.425817 28183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:39.428498 28251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:39.434156 28251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8637d5a710e46ae979d69f9d276e3d9: Bootstrap starting.
I20260926 11:44:39.435302 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 28183
I20260926 11:44:39.435442 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:39.435704 6528 external_mini_cluster.cc:1428] Setting key 7c3ec39780fba9d063b364d98d87e625
I20260926 11:44:39.436887 28251 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8637d5a710e46ae979d69f9d276e3d9: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:39.437729 28251 log.cc:824] T 00000000000000000000000000000000 P d8637d5a710e46ae979d69f9d276e3d9: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:39.439844 28251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8637d5a710e46ae979d69f9d276e3d9: No bootstrap required, opened a new log
I20260926 11:44:39.443862 28251 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8637d5a710e46ae979d69f9d276e3d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8637d5a710e46ae979d69f9d276e3d9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45763 } }
I20260926 11:44:39.444128 28251 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8637d5a710e46ae979d69f9d276e3d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:39.444186 28251 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8637d5a710e46ae979d69f9d276e3d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8637d5a710e46ae979d69f9d276e3d9, State: Initialized, Role: FOLLOWER
I20260926 11:44:39.444648 28251 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8637d5a710e46ae979d69f9d276e3d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8637d5a710e46ae979d69f9d276e3d9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45763 } }
I20260926 11:44:39.444797 28251 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8637d5a710e46ae979d69f9d276e3d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:44:39.444862 28251 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8637d5a710e46ae979d69f9d276e3d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:44:39.444950 28251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8637d5a710e46ae979d69f9d276e3d9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:39.445046 28254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:39.436740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54237 (local address 127.6.96.62:45763)
0926 11:44:39.437151 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:39.437169 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:39.437209 (+ 40us) server_negotiation.cc:407] Connection header received
0926 11:44:39.438025 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:39.438052 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:39.438415 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:39.438755 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:39.439185 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:39.440575 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:39.441523 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:39.442235 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:39.442399 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:39.443151 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:39.443157 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:39.443174 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:39.443646 (+ 472us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:39.443710 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:39.443737 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:39.443874 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:39.443989 (+ 115us) server_negotiation.cc:299] Negotiation successful
0926 11:44:39.444115 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":118,"threads_started":1}
I20260926 11:44:39.445828 28251 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8637d5a710e46ae979d69f9d276e3d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8637d5a710e46ae979d69f9d276e3d9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45763 } }
I20260926 11:44:39.446185 28251 leader_election.cc:302] T 00000000000000000000000000000000 P d8637d5a710e46ae979d69f9d276e3d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8637d5a710e46ae979d69f9d276e3d9; no voters:
I20260926 11:44:39.446519 28251 leader_election.cc:288] T 00000000000000000000000000000000 P d8637d5a710e46ae979d69f9d276e3d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:44:39.446620 28256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8637d5a710e46ae979d69f9d276e3d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:39.446877 28256 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8637d5a710e46ae979d69f9d276e3d9 [term 1 LEADER]: Becoming Leader. State: Replica: d8637d5a710e46ae979d69f9d276e3d9, State: Running, Role: LEADER
I20260926 11:44:39.447281 28256 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8637d5a710e46ae979d69f9d276e3d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8637d5a710e46ae979d69f9d276e3d9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45763 } }
I20260926 11:44:39.447791 28251 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8637d5a710e46ae979d69f9d276e3d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:39.447914 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 11:44:39.448773 28257 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8637d5a710e46ae979d69f9d276e3d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8637d5a710e46ae979d69f9d276e3d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8637d5a710e46ae979d69f9d276e3d9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45763 } } }
I20260926 11:44:39.448889 28257 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8637d5a710e46ae979d69f9d276e3d9 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:39.449090 28257 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8637d5a710e46ae979d69f9d276e3d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8637d5a710e46ae979d69f9d276e3d9. Latest consensus state: current_term: 1 leader_uuid: "d8637d5a710e46ae979d69f9d276e3d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8637d5a710e46ae979d69f9d276e3d9" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45763 } } }
I20260926 11:44:39.449172 28257 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8637d5a710e46ae979d69f9d276e3d9 [sys.catalog]: This master's current role is: LEADER
W20260926 11:44:39.453039 28275 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d8637d5a710e46ae979d69f9d276e3d9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 11:44:39.453121 28275 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 11:44:39.453234 28265 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:39.454092 28265 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:39.454542 28254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:39.450153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55893 (local address 127.6.96.62:45763)
0926 11:44:39.450276 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:39.450281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:39.450462 (+ 181us) server_negotiation.cc:407] Connection header received
0926 11:44:39.450572 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:39.450574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:39.450616 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:39.450697 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:39.450955 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:39.451680 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:39.452793 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:39.453125 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:39.453577 (+ 452us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:39.454063 (+ 486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:39.454067 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:39.454070 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:39.454081 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:39.454104 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:39.454114 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:39.454199 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:39.454383 (+ 184us) server_negotiation.cc:299] Negotiation successful
0926 11:44:39.454415 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 11:44:39.459565 28265 catalog_manager.cc:1347] Generated new cluster ID: 18a435b2456c44b2a7e869943323208e
I20260926 11:44:39.459618 28265 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:39.469439 28265 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:39.470546 28265 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:39.477300 28265 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8637d5a710e46ae979d69f9d276e3d9: Generated new TSK 0
I20260926 11:44:39.478130 28265 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:44:39.489424 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 28183
W20260926 11:44:39.493878 28168 connection.cc:535] client connection to 127.6.96.62:45763 recv error: Network error: failed to read from TLS socket (remote: 127.6.96.62:45763): Connection reset by peer (error 104)
2026-09-26T11:44:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1306 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:44:39 dist-test-slave-xxxx krb5kdc[28282](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:44:39 dist-test-slave-xxxx krb5kdc[28282](info): set up 2 sockets
Sep 26 11:44:39 dist-test-slave-xxxx krb5kdc[28282](info): commencing operation
krb5kdc: starting...
W20260926 11:44:40.438851 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.931s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:44:40 dist-test-slave-xxxx krb5kdc[28282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:44:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:44:40.549996 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:45249
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:38319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:45249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:40.655857 28295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:40.656136 28295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:40.656201 28295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:40.660321 28295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:40.660393 28295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:40.660420 28295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:40.660441 28295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:40.660462 28295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:40.665776 28295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:38319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:45249
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:45249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:40.666867 28295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:40.667874 28295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 11:44:40.676280 28295 server_base.cc:1034] running on GCE node
W20260926 11:44:40.676199 28301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:40.676199 28303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:40.676205 28300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:40.677001 28295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:40.678099 28295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:40.679275 28295 hybrid_clock.cc:648] HybridClock initialized: now 1790423080679256 us; error 34 us; skew 500 ppm
Sep 26 11:44:40 dist-test-slave-xxxx krb5kdc[28282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423080, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:40.691897 28295 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:44:40.692900 28295 webserver.cc:460] Webserver started at http://127.6.96.62:42625/ using document root <none> and password file <none>
I20260926 11:44:40.693471 28295 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:40.693534 28295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:40.693738 28295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:40.695851 28295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "4c7f25a6f41c4d9096d17c9dc0770d2e"
format_stamp: "Formatted at 2026-09-26 11:44:40 on dist-test-slave-xxxx"
server_key: "7ecdef4190eb6973f1d3f415766ef16d"
server_key_iv: "10eec886729115ded0607b2d905fa369"
server_key_version: "encryptionkey@0"
I20260926 11:44:40.696365 28295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "4c7f25a6f41c4d9096d17c9dc0770d2e"
format_stamp: "Formatted at 2026-09-26 11:44:40 on dist-test-slave-xxxx"
server_key: "7ecdef4190eb6973f1d3f415766ef16d"
server_key_iv: "10eec886729115ded0607b2d905fa369"
server_key_version: "encryptionkey@0"
I20260926 11:44:40.699994 28295 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260926 11:44:40.702409 28310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:40.703598 28295 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:44:40.703714 28295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "4c7f25a6f41c4d9096d17c9dc0770d2e"
format_stamp: "Formatted at 2026-09-26 11:44:40 on dist-test-slave-xxxx"
server_key: "7ecdef4190eb6973f1d3f415766ef16d"
server_key_iv: "10eec886729115ded0607b2d905fa369"
server_key_version: "encryptionkey@0"
I20260926 11:44:40.703814 28295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:40.708218 28295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:40.711196 28295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:40.711364 28295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:40.717877 28295 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:45249
I20260926 11:44:40.717964 28362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:45249 every 8 connection(s)
I20260926 11:44:40.719008 28295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:40.723537 28363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:40.725507 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 28295
I20260926 11:44:40.725653 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:40.725937 6528 external_mini_cluster.cc:1428] Setting key 54e7c56bbac14359dbf9de3f5c44db47
I20260926 11:44:40.729466 28363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c7f25a6f41c4d9096d17c9dc0770d2e: Bootstrap starting.
I20260926 11:44:40.731374 28366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:40.727377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39747 (local address 127.6.96.62:45249)
0926 11:44:40.727763 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:40.727779 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:40.727813 (+ 34us) server_negotiation.cc:407] Connection header received
0926 11:44:40.728655 (+ 842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:40.728675 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:40.728963 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:40.729288 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:40.730048 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:40.730054 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:40.730070 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:40.730081 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:40.730144 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:40.730168 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:40.730303 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:40.730370 (+ 67us) server_negotiation.cc:299] Negotiation successful
0926 11:44:40.730623 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":103,"threads_started":1}
I20260926 11:44:40.732019 28363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c7f25a6f41c4d9096d17c9dc0770d2e: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:40.732720 28363 log.cc:824] T 00000000000000000000000000000000 P 4c7f25a6f41c4d9096d17c9dc0770d2e: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:40.733918 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 11:44:40.734890 28363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c7f25a6f41c4d9096d17c9dc0770d2e: No bootstrap required, opened a new log
I20260926 11:44:40.737540 28366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:40.736080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36407 (local address 127.6.96.62:45249)
0926 11:44:40.736192 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:40.736197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:40.736400 (+ 203us) server_negotiation.cc:407] Connection header received
0926 11:44:40.736478 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:40.736481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:40.736514 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:40.736591 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:40.737192 (+ 601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:40.737196 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:40.737199 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:40.737210 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:40.737245 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:40.737258 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:40.737343 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:40.737390 (+ 47us) server_negotiation.cc:299] Negotiation successful
0926 11:44:40.737426 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 11:44:40.739009 28363 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c7f25a6f41c4d9096d17c9dc0770d2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7f25a6f41c4d9096d17c9dc0770d2e" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45249 } }
I20260926 11:44:40.739221 28363 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c7f25a6f41c4d9096d17c9dc0770d2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:40.739279 28363 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c7f25a6f41c4d9096d17c9dc0770d2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7f25a6f41c4d9096d17c9dc0770d2e, State: Initialized, Role: FOLLOWER
I20260926 11:44:40.739676 28363 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c7f25a6f41c4d9096d17c9dc0770d2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7f25a6f41c4d9096d17c9dc0770d2e" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45249 } }
I20260926 11:44:40.739804 28363 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c7f25a6f41c4d9096d17c9dc0770d2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:44:40.739866 28363 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c7f25a6f41c4d9096d17c9dc0770d2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:44:40.740020 28363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c7f25a6f41c4d9096d17c9dc0770d2e [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:40.740916 28363 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c7f25a6f41c4d9096d17c9dc0770d2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7f25a6f41c4d9096d17c9dc0770d2e" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45249 } }
I20260926 11:44:40.741243 28363 leader_election.cc:302] T 00000000000000000000000000000000 P 4c7f25a6f41c4d9096d17c9dc0770d2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c7f25a6f41c4d9096d17c9dc0770d2e; no voters:
I20260926 11:44:40.741518 28363 leader_election.cc:288] T 00000000000000000000000000000000 P 4c7f25a6f41c4d9096d17c9dc0770d2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:44:40.741647 28373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c7f25a6f41c4d9096d17c9dc0770d2e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:40.741848 28373 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c7f25a6f41c4d9096d17c9dc0770d2e [term 1 LEADER]: Becoming Leader. State: Replica: 4c7f25a6f41c4d9096d17c9dc0770d2e, State: Running, Role: LEADER
I20260926 11:44:40.742137 28373 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c7f25a6f41c4d9096d17c9dc0770d2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7f25a6f41c4d9096d17c9dc0770d2e" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45249 } }
I20260926 11:44:40.742698 28363 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c7f25a6f41c4d9096d17c9dc0770d2e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:40.743677 28374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c7f25a6f41c4d9096d17c9dc0770d2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c7f25a6f41c4d9096d17c9dc0770d2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7f25a6f41c4d9096d17c9dc0770d2e" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45249 } } }
I20260926 11:44:40.743805 28374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c7f25a6f41c4d9096d17c9dc0770d2e [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:40.744144 28375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c7f25a6f41c4d9096d17c9dc0770d2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c7f25a6f41c4d9096d17c9dc0770d2e. Latest consensus state: current_term: 1 leader_uuid: "4c7f25a6f41c4d9096d17c9dc0770d2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7f25a6f41c4d9096d17c9dc0770d2e" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45249 } } }
I20260926 11:44:40.744278 28375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c7f25a6f41c4d9096d17c9dc0770d2e [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:40.744316 28382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:40.747502 28382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:40.752961 28382 catalog_manager.cc:1347] Generated new cluster ID: d4019acff5534b12b2e4a6eb7f35c607
I20260926 11:44:40.753027 28382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:40.773464 28382 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:40.773815 28382 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:40.777695 28382 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c7f25a6f41c4d9096d17c9dc0770d2e: Generated new TSK 0
I20260926 11:44:40.778486 28382 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:44:40.785516 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 28295
2026-09-26T11:44:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1296 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:44:40 dist-test-slave-xxxx krb5kdc[28394](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:44:40 dist-test-slave-xxxx krb5kdc[28394](info): set up 2 sockets
Sep 26 11:44:40 dist-test-slave-xxxx krb5kdc[28394](info): commencing operation
krb5kdc: starting...
W20260926 11:44:41.730926 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:44:41 dist-test-slave-xxxx krb5kdc[28394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:44:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:44:41.847442 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:46339
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:46427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:46339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:41.954878 28407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:41.955159 28407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:41.955217 28407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:41.959370 28407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:41.959443 28407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:41.959470 28407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:41.959493 28407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:41.959522 28407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:41.965288 28407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:46427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:46339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:46339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:41.966583 28407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:41.967717 28407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:41.976689 28413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:41.976722 28415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:41.976817 28407 server_base.cc:1034] running on GCE node
W20260926 11:44:41.976688 28412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:41.977623 28407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:41.978850 28407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:41.980031 28407 hybrid_clock.cc:648] HybridClock initialized: now 1790423081980011 us; error 33 us; skew 500 ppm
Sep 26 11:44:41 dist-test-slave-xxxx krb5kdc[28394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423081, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:41.982810 28407 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:44:41.983697 28407 webserver.cc:460] Webserver started at http://127.6.96.62:35489/ using document root <none> and password file <none>
I20260926 11:44:41.984211 28407 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:41.984269 28407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:41.984441 28407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:41.986449 28407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "b2c05c8ffeb045bc8abc3ca3beb5ee1a"
format_stamp: "Formatted at 2026-09-26 11:44:41 on dist-test-slave-xxxx"
server_key: "3d3ca1a7508aef7f61e6b598ed942249"
server_key_iv: "e4acc625db8f74f773ad8338cd95f30d"
server_key_version: "encryptionkey@0"
I20260926 11:44:41.987002 28407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "b2c05c8ffeb045bc8abc3ca3beb5ee1a"
format_stamp: "Formatted at 2026-09-26 11:44:41 on dist-test-slave-xxxx"
server_key: "3d3ca1a7508aef7f61e6b598ed942249"
server_key_iv: "e4acc625db8f74f773ad8338cd95f30d"
server_key_version: "encryptionkey@0"
I20260926 11:44:41.990703 28407 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:44:41.993149 28422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:41.994377 28407 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:44:41.994493 28407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "b2c05c8ffeb045bc8abc3ca3beb5ee1a"
format_stamp: "Formatted at 2026-09-26 11:44:41 on dist-test-slave-xxxx"
server_key: "3d3ca1a7508aef7f61e6b598ed942249"
server_key_iv: "e4acc625db8f74f773ad8338cd95f30d"
server_key_version: "encryptionkey@0"
I20260926 11:44:41.994593 28407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:42.005672 28407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:42.008647 28407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:42.008831 28407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:42.015283 28407 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:46339
I20260926 11:44:42.015296 28474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:46339 every 8 connection(s)
I20260926 11:44:42.016152 28407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:42.018803 28475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:42.023329 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 28407
I20260926 11:44:42.023473 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:42.023736 6528 external_mini_cluster.cc:1428] Setting key 17168b8d7aa0c5554bcc9fb2c7be0863
I20260926 11:44:42.024390 28475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2c05c8ffeb045bc8abc3ca3beb5ee1a: Bootstrap starting.
I20260926 11:44:42.027123 28475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2c05c8ffeb045bc8abc3ca3beb5ee1a: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:42.027932 28475 log.cc:824] T 00000000000000000000000000000000 P b2c05c8ffeb045bc8abc3ca3beb5ee1a: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:42.029953 28475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2c05c8ffeb045bc8abc3ca3beb5ee1a: No bootstrap required, opened a new log
Sep 26 11:44:42 dist-test-slave-xxxx krb5kdc[28394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:44:42.034309 28475 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2c05c8ffeb045bc8abc3ca3beb5ee1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c05c8ffeb045bc8abc3ca3beb5ee1a" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46339 } }
I20260926 11:44:42.034590 28475 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2c05c8ffeb045bc8abc3ca3beb5ee1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:42.034651 28475 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2c05c8ffeb045bc8abc3ca3beb5ee1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c05c8ffeb045bc8abc3ca3beb5ee1a, State: Initialized, Role: FOLLOWER
I20260926 11:44:42.035168 28475 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2c05c8ffeb045bc8abc3ca3beb5ee1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c05c8ffeb045bc8abc3ca3beb5ee1a" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46339 } }
I20260926 11:44:42.035364 28475 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2c05c8ffeb045bc8abc3ca3beb5ee1a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:44:42.035430 28475 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2c05c8ffeb045bc8abc3ca3beb5ee1a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:44:42.035523 28475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2c05c8ffeb045bc8abc3ca3beb5ee1a [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:42.036566 28475 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2c05c8ffeb045bc8abc3ca3beb5ee1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c05c8ffeb045bc8abc3ca3beb5ee1a" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46339 } }
I20260926 11:44:42.036921 28475 leader_election.cc:302] T 00000000000000000000000000000000 P b2c05c8ffeb045bc8abc3ca3beb5ee1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2c05c8ffeb045bc8abc3ca3beb5ee1a; no voters:
I20260926 11:44:42.037238 28475 leader_election.cc:288] T 00000000000000000000000000000000 P b2c05c8ffeb045bc8abc3ca3beb5ee1a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:44:42.037326 28480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2c05c8ffeb045bc8abc3ca3beb5ee1a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:42.037750 28480 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2c05c8ffeb045bc8abc3ca3beb5ee1a [term 1 LEADER]: Becoming Leader. State: Replica: b2c05c8ffeb045bc8abc3ca3beb5ee1a, State: Running, Role: LEADER
I20260926 11:44:42.038087 28480 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2c05c8ffeb045bc8abc3ca3beb5ee1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c05c8ffeb045bc8abc3ca3beb5ee1a" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46339 } }
I20260926 11:44:42.038527 28475 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2c05c8ffeb045bc8abc3ca3beb5ee1a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:42.038542 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:42.025033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53887 (local address 127.6.96.62:46339)
0926 11:44:42.025437 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:42.025461 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:42.025505 (+ 44us) server_negotiation.cc:407] Connection header received
0926 11:44:42.026354 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:42.026379 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:42.026813 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:42.027231 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:42.027931 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:42.029322 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:42.030197 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:42.030882 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:42.031030 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:42.033623 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:42.033630 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:42.033647 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:42.033678 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:42.035486 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:42.035908 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:42.035914 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:42.035920 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:42.036046 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:42.036260 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:42.036263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:42.036265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:42.036778 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:42.036959 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:42.037311 (+ 352us) server_negotiation.cc:299] Negotiation successful
0926 11:44:42.037607 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":260,"thread_start_us":105,"threads_started":1}
I20260926 11:44:42.039753 28481 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2c05c8ffeb045bc8abc3ca3beb5ee1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2c05c8ffeb045bc8abc3ca3beb5ee1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c05c8ffeb045bc8abc3ca3beb5ee1a" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46339 } } }
I20260926 11:44:42.039750 28482 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2c05c8ffeb045bc8abc3ca3beb5ee1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2c05c8ffeb045bc8abc3ca3beb5ee1a. Latest consensus state: current_term: 1 leader_uuid: "b2c05c8ffeb045bc8abc3ca3beb5ee1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c05c8ffeb045bc8abc3ca3beb5ee1a" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 46339 } } }
I20260926 11:44:42.039893 28482 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2c05c8ffeb045bc8abc3ca3beb5ee1a [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:42.039893 28481 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2c05c8ffeb045bc8abc3ca3beb5ee1a [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:42.041674 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260926 11:44:42.043268 28495 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b2c05c8ffeb045bc8abc3ca3beb5ee1a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 11:44:42.043331 28495 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 11:44:42.043386 28490 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:42.045323 28490 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:42.052103 28490 catalog_manager.cc:1347] Generated new cluster ID: 207f84986911461182ee2fbba8ede9ae
I20260926 11:44:42.052191 28490 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:42.052296 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:42.043696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34619 (local address 127.6.96.62:46339)
0926 11:44:42.043804 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:42.043808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:42.044410 (+ 602us) server_negotiation.cc:407] Connection header received
0926 11:44:42.044513 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:42.044516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:42.044562 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:42.044642 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:42.045403 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:42.046149 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:42.047268 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:42.047623 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:42.047735 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:42.049023 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:42.049027 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:42.049030 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:42.049054 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:42.050428 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:42.050969 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:42.050972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:42.050974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:42.051062 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:42.051338 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:42.051342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:42.051343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:42.051545 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:42.051666 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:42.051949 (+ 283us) server_negotiation.cc:299] Negotiation successful
0926 11:44:42.052065 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 11:44:42.061717 28490 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:42.062886 28490 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:42.072669 28490 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2c05c8ffeb045bc8abc3ca3beb5ee1a: Generated new TSK 0
I20260926 11:44:42.073338 28490 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:44:42.079891 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 28407
2026-09-26T11:44:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1294 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:44:42 dist-test-slave-xxxx krb5kdc[28506](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:44:42 dist-test-slave-xxxx krb5kdc[28506](info): set up 2 sockets
Sep 26 11:44:42 dist-test-slave-xxxx krb5kdc[28506](info): commencing operation
krb5kdc: starting...
W20260926 11:44:43.030694 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:44:43 dist-test-slave-xxxx krb5kdc[28506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:44:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:44:43.150115 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:45293
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:42591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:45293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:43.260860 28519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:43.261168 28519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:43.261229 28519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:43.265341 28519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:43.265412 28519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:43.265439 28519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:43.265460 28519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:43.265481 28519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:43.271183 28519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:42591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:45293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:45293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:43.272291 28519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:43.273332 28519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:43.281684 28524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:43.281769 28525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:43.281684 28527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:43.282037 28519 server_base.cc:1034] running on GCE node
I20260926 11:44:43.282595 28519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:43.283762 28519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:43.284927 28519 hybrid_clock.cc:648] HybridClock initialized: now 1790423083284910 us; error 30 us; skew 500 ppm
Sep 26 11:44:43 dist-test-slave-xxxx krb5kdc[28506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423083, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:43.287647 28519 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:44:43.288563 28519 webserver.cc:460] Webserver started at http://127.6.96.62:39549/ using document root <none> and password file <none>
I20260926 11:44:43.289072 28519 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:43.289129 28519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:43.289296 28519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:43.291327 28519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "8131adbee47344acaa88f5bf9c707f0c"
format_stamp: "Formatted at 2026-09-26 11:44:43 on dist-test-slave-xxxx"
server_key: "d85637a6f6bfe2f5a66d5301690423d3"
server_key_iv: "eb33e46357f36f0994cd90dbe486912a"
server_key_version: "encryptionkey@0"
I20260926 11:44:43.291844 28519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "8131adbee47344acaa88f5bf9c707f0c"
format_stamp: "Formatted at 2026-09-26 11:44:43 on dist-test-slave-xxxx"
server_key: "d85637a6f6bfe2f5a66d5301690423d3"
server_key_iv: "eb33e46357f36f0994cd90dbe486912a"
server_key_version: "encryptionkey@0"
I20260926 11:44:43.295420 28519 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260926 11:44:43.297947 28534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:43.299376 28519 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 11:44:43.299496 28519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "8131adbee47344acaa88f5bf9c707f0c"
format_stamp: "Formatted at 2026-09-26 11:44:43 on dist-test-slave-xxxx"
server_key: "d85637a6f6bfe2f5a66d5301690423d3"
server_key_iv: "eb33e46357f36f0994cd90dbe486912a"
server_key_version: "encryptionkey@0"
I20260926 11:44:43.299598 28519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:43.326893 28519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:43.330175 28519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:43.330366 28519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:43.337263 28519 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:45293
I20260926 11:44:43.337280 28586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:45293 every 8 connection(s)
I20260926 11:44:43.338234 28519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:43.341058 28587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:43.346529 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 28519
I20260926 11:44:43.346676 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:43.346989 6528 external_mini_cluster.cc:1428] Setting key f27c1d8cdc95c8df8c47792b432e09f9
I20260926 11:44:43.347047 28587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8131adbee47344acaa88f5bf9c707f0c: Bootstrap starting.
I20260926 11:44:43.349601 28587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8131adbee47344acaa88f5bf9c707f0c: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:43.350447 28587 log.cc:824] T 00000000000000000000000000000000 P 8131adbee47344acaa88f5bf9c707f0c: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:43.352599 28587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8131adbee47344acaa88f5bf9c707f0c: No bootstrap required, opened a new log
I20260926 11:44:43.355886 28590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:43.348335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51503 (local address 127.6.96.62:45293)
0926 11:44:43.348747 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:43.348762 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:43.348798 (+ 36us) server_negotiation.cc:407] Connection header received
0926 11:44:43.349439 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:43.349457 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:43.349883 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:43.350181 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:43.350513 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:43.351568 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:43.352728 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:43.353205 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:43.353319 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:43.354187 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:43.354192 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:43.354205 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:43.354590 (+ 385us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:43.354645 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:43.354666 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:43.354738 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:43.354919 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 11:44:43.355166 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":116,"threads_started":1}
I20260926 11:44:43.356501 28587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8131adbee47344acaa88f5bf9c707f0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8131adbee47344acaa88f5bf9c707f0c" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45293 } }
I20260926 11:44:43.356729 28587 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8131adbee47344acaa88f5bf9c707f0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:43.356772 28587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8131adbee47344acaa88f5bf9c707f0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8131adbee47344acaa88f5bf9c707f0c, State: Initialized, Role: FOLLOWER
I20260926 11:44:43.357214 28587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8131adbee47344acaa88f5bf9c707f0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8131adbee47344acaa88f5bf9c707f0c" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45293 } }
I20260926 11:44:43.357343 28587 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8131adbee47344acaa88f5bf9c707f0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:44:43.357405 28587 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8131adbee47344acaa88f5bf9c707f0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:44:43.357501 28587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8131adbee47344acaa88f5bf9c707f0c [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:43.358476 28587 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8131adbee47344acaa88f5bf9c707f0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8131adbee47344acaa88f5bf9c707f0c" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45293 } }
I20260926 11:44:43.358518 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 11:44:43.358867 28587 leader_election.cc:302] T 00000000000000000000000000000000 P 8131adbee47344acaa88f5bf9c707f0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8131adbee47344acaa88f5bf9c707f0c; no voters:
I20260926 11:44:43.359200 28587 leader_election.cc:288] T 00000000000000000000000000000000 P 8131adbee47344acaa88f5bf9c707f0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:44:43.359284 28592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8131adbee47344acaa88f5bf9c707f0c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:43.359488 28592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8131adbee47344acaa88f5bf9c707f0c [term 1 LEADER]: Becoming Leader. State: Replica: 8131adbee47344acaa88f5bf9c707f0c, State: Running, Role: LEADER
I20260926 11:44:43.359854 28592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8131adbee47344acaa88f5bf9c707f0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8131adbee47344acaa88f5bf9c707f0c" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45293 } }
I20260926 11:44:43.360527 28587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8131adbee47344acaa88f5bf9c707f0c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:43.361334 28593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8131adbee47344acaa88f5bf9c707f0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8131adbee47344acaa88f5bf9c707f0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8131adbee47344acaa88f5bf9c707f0c" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45293 } } }
I20260926 11:44:43.361464 28593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8131adbee47344acaa88f5bf9c707f0c [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:43.361789 28594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8131adbee47344acaa88f5bf9c707f0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8131adbee47344acaa88f5bf9c707f0c. Latest consensus state: current_term: 1 leader_uuid: "8131adbee47344acaa88f5bf9c707f0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8131adbee47344acaa88f5bf9c707f0c" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 45293 } } }
I20260926 11:44:43.361864 28594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8131adbee47344acaa88f5bf9c707f0c [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:43.362191 28606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:43.365224 28590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:43.360703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58399 (local address 127.6.96.62:45293)
0926 11:44:43.360805 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:43.360809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:43.361047 (+ 238us) server_negotiation.cc:407] Connection header received
0926 11:44:43.361162 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:43.361165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:43.361214 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:43.361294 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:43.361685 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:43.362409 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:43.363485 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:43.363841 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:43.363933 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:43.364811 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:43.364814 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:43.364831 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:43.364844 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:43.364871 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:43.364883 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:43.364942 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:43.365050 (+ 108us) server_negotiation.cc:299] Negotiation successful
0926 11:44:43.365097 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":39}
I20260926 11:44:43.365909 28606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:43.371389 28606 catalog_manager.cc:1347] Generated new cluster ID: ea03971193f34a12a3060da0ed426884
I20260926 11:44:43.371451 28606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:43.379068 28606 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:43.380290 28606 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:43.388976 28606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8131adbee47344acaa88f5bf9c707f0c: Generated new TSK 0
I20260926 11:44:43.389864 28606 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:44:43.394707 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 28519
2026-09-26T11:44:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1314 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:44:43 dist-test-slave-xxxx krb5kdc[28618](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:44:43 dist-test-slave-xxxx krb5kdc[28618](info): set up 2 sockets
Sep 26 11:44:43 dist-test-slave-xxxx krb5kdc[28618](info): commencing operation
krb5kdc: starting...
W20260926 11:44:44.334900 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.922s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:44:44 dist-test-slave-xxxx krb5kdc[28618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:44:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:44:44.453933 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:33535
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:36437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:33535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:44.564061 28631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:44.564353 28631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:44.564414 28631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:44.568583 28631 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:44.568668 28631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:44.568696 28631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:44.568718 28631 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:44.568743 28631 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:44.574219 28631 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:36437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:33535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:33535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:44.575400 28631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:44.576440 28631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:44.584903 28636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:44.584903 28639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:44.584913 28637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:44.585266 28631 server_base.cc:1034] running on GCE node
I20260926 11:44:44.585896 28631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:44.587074 28631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:44.588227 28631 hybrid_clock.cc:648] HybridClock initialized: now 1790423084588199 us; error 42 us; skew 500 ppm
Sep 26 11:44:44 dist-test-slave-xxxx krb5kdc[28618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423084, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:44.591063 28631 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:44:44.591976 28631 webserver.cc:460] Webserver started at http://127.6.96.62:40661/ using document root <none> and password file <none>
I20260926 11:44:44.592530 28631 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:44.592589 28631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:44.592794 28631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:44.594880 28631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "8e6db076ee7546d7b403e5cc08b757d4"
format_stamp: "Formatted at 2026-09-26 11:44:44 on dist-test-slave-xxxx"
server_key: "6117b3420f03c7c62240843aef908a85"
server_key_iv: "2917ea08b87dd1a1e82c4951828c75a6"
server_key_version: "encryptionkey@0"
I20260926 11:44:44.595429 28631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "8e6db076ee7546d7b403e5cc08b757d4"
format_stamp: "Formatted at 2026-09-26 11:44:44 on dist-test-slave-xxxx"
server_key: "6117b3420f03c7c62240843aef908a85"
server_key_iv: "2917ea08b87dd1a1e82c4951828c75a6"
server_key_version: "encryptionkey@0"
I20260926 11:44:44.599095 28631 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 11:44:44.601533 28646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:44.602818 28631 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 11:44:44.602972 28631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "8e6db076ee7546d7b403e5cc08b757d4"
format_stamp: "Formatted at 2026-09-26 11:44:44 on dist-test-slave-xxxx"
server_key: "6117b3420f03c7c62240843aef908a85"
server_key_iv: "2917ea08b87dd1a1e82c4951828c75a6"
server_key_version: "encryptionkey@0"
I20260926 11:44:44.603092 28631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:44.607820 28631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:44.610924 28631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:44.611110 28631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:44.617786 28631 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:33535
I20260926 11:44:44.617811 28698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:33535 every 8 connection(s)
I20260926 11:44:44.618705 28631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:44.619433 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 28631
I20260926 11:44:44.619554 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:44.619815 6528 external_mini_cluster.cc:1428] Setting key 4b3d99682529edec086aae10c5baa0af
I20260926 11:44:44.622679 28699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:44.624830 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:44.621099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54155 (local address 127.6.96.62:33535)
0926 11:44:44.621492 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:44.621506 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:44.621542 (+ 36us) server_negotiation.cc:407] Connection header received
0926 11:44:44.622173 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:44.622190 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:44.622505 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:44.622820 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:44.623638 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:44.623642 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:44.623655 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:44.623664 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:44.623715 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:44.623735 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:44.623838 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:44.623933 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 11:44:44.624085 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":115,"threads_started":1}
I20260926 11:44:44.627760 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 11:44:44.629169 28699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e6db076ee7546d7b403e5cc08b757d4: Bootstrap starting.
I20260926 11:44:44.631327 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:44.629702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58119 (local address 127.6.96.62:33535)
0926 11:44:44.629816 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:44.629820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:44.630063 (+ 243us) server_negotiation.cc:407] Connection header received
0926 11:44:44.630182 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:44.630185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:44.630215 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:44.630281 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:44.630961 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:44.630964 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:44.630966 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 11:44:44.630976 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:44.631009 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 11:44:44.631020 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:44.631087 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:44.631170 (+ 83us) server_negotiation.cc:299] Negotiation successful
0926 11:44:44.631201 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 11:44:44.632037 28699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e6db076ee7546d7b403e5cc08b757d4: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:44.632848 28699 log.cc:824] T 00000000000000000000000000000000 P 8e6db076ee7546d7b403e5cc08b757d4: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:44.635061 28699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e6db076ee7546d7b403e5cc08b757d4: No bootstrap required, opened a new log
I20260926 11:44:44.638654 28699 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e6db076ee7546d7b403e5cc08b757d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e6db076ee7546d7b403e5cc08b757d4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 33535 } }
I20260926 11:44:44.638933 28699 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e6db076ee7546d7b403e5cc08b757d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 11:44:44.638988 28699 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e6db076ee7546d7b403e5cc08b757d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e6db076ee7546d7b403e5cc08b757d4, State: Initialized, Role: FOLLOWER
I20260926 11:44:44.639477 28699 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e6db076ee7546d7b403e5cc08b757d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e6db076ee7546d7b403e5cc08b757d4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 33535 } }
I20260926 11:44:44.639624 28699 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e6db076ee7546d7b403e5cc08b757d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 11:44:44.639691 28699 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e6db076ee7546d7b403e5cc08b757d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 11:44:44.639787 28699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e6db076ee7546d7b403e5cc08b757d4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:44.640892 28699 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e6db076ee7546d7b403e5cc08b757d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e6db076ee7546d7b403e5cc08b757d4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 33535 } }
I20260926 11:44:44.641283 28699 leader_election.cc:302] T 00000000000000000000000000000000 P 8e6db076ee7546d7b403e5cc08b757d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e6db076ee7546d7b403e5cc08b757d4; no voters:
I20260926 11:44:44.641634 28699 leader_election.cc:288] T 00000000000000000000000000000000 P 8e6db076ee7546d7b403e5cc08b757d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 11:44:44.641743 28709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e6db076ee7546d7b403e5cc08b757d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:44.641953 28709 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e6db076ee7546d7b403e5cc08b757d4 [term 1 LEADER]: Becoming Leader. State: Replica: 8e6db076ee7546d7b403e5cc08b757d4, State: Running, Role: LEADER
I20260926 11:44:44.642297 28709 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e6db076ee7546d7b403e5cc08b757d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e6db076ee7546d7b403e5cc08b757d4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 33535 } }
I20260926 11:44:44.642875 28699 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e6db076ee7546d7b403e5cc08b757d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:44.644194 28710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e6db076ee7546d7b403e5cc08b757d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e6db076ee7546d7b403e5cc08b757d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e6db076ee7546d7b403e5cc08b757d4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 33535 } } }
I20260926 11:44:44.644392 28710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e6db076ee7546d7b403e5cc08b757d4 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:44.644194 28711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e6db076ee7546d7b403e5cc08b757d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e6db076ee7546d7b403e5cc08b757d4. Latest consensus state: current_term: 1 leader_uuid: "8e6db076ee7546d7b403e5cc08b757d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e6db076ee7546d7b403e5cc08b757d4" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 33535 } } }
I20260926 11:44:44.644809 28718 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:44.648034 28718 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:44.649663 28711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e6db076ee7546d7b403e5cc08b757d4 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:44.653672 28718 catalog_manager.cc:1347] Generated new cluster ID: 95a42f33accd429a879ee0757a76b9ac
I20260926 11:44:44.653744 28718 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:44.665635 28718 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:44.665999 28718 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:44.675490 28718 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e6db076ee7546d7b403e5cc08b757d4: Generated new TSK 0
I20260926 11:44:44.676329 28718 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 11:44:44.685914 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 28631
2026-09-26T11:44:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1290 ms)
[----------] 18 tests from AuthTokenIssuingTest (23639 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:44:44 dist-test-slave-xxxx krb5kdc[28730](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:44:44 dist-test-slave-xxxx krb5kdc[28730](info): set up 2 sockets
Sep 26 11:44:44 dist-test-slave-xxxx krb5kdc[28730](info): commencing operation
krb5kdc: starting...
W20260926 11:44:45.658996 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.955s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:44:45 dist-test-slave-xxxx krb5kdc[28730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:44:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:44:45.778715 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:32771
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:33701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:32771,127.6.96.61:38979,127.6.96.60:44137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:45.885799 28743 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 11:44:45.886107 28743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:45.886132 28743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:45.886175 28743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:45.890166 28743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:45.890228 28743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:45.890246 28743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:45.890261 28743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:45.890275 28743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:45.895509 28743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:33701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:32771,127.6.96.61:38979,127.6.96.60:44137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:32771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:45.896579 28743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:45.897588 28743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:45.905821 28748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:45.905892 28743 server_base.cc:1034] running on GCE node
W20260926 11:44:45.905815 28751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:45.905825 28749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:45.906618 28743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:45.907699 28743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:45.908856 28743 hybrid_clock.cc:648] HybridClock initialized: now 1790423085908839 us; error 27 us; skew 500 ppm
Sep 26 11:44:45 dist-test-slave-xxxx krb5kdc[28730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423085, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:45.917596 28743 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:44:45.918522 28743 webserver.cc:460] Webserver started at http://127.6.96.62:38383/ using document root <none> and password file <none>
I20260926 11:44:45.919118 28743 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:45.919183 28743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:45.919381 28743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:45.921412 28743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6"
format_stamp: "Formatted at 2026-09-26 11:44:45 on dist-test-slave-xxxx"
server_key: "5b2e396b852d9cf8fa599be274e25267"
server_key_iv: "e9bbc7b7ec30c2f681056f924e8474a0"
server_key_version: "encryptionkey@0"
I20260926 11:44:45.921949 28743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6"
format_stamp: "Formatted at 2026-09-26 11:44:45 on dist-test-slave-xxxx"
server_key: "5b2e396b852d9cf8fa599be274e25267"
server_key_iv: "e9bbc7b7ec30c2f681056f924e8474a0"
server_key_version: "encryptionkey@0"
I20260926 11:44:45.925539 28743 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 11:44:45.927948 28758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:45.929144 28743 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 11:44:45.929255 28743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6"
format_stamp: "Formatted at 2026-09-26 11:44:45 on dist-test-slave-xxxx"
server_key: "5b2e396b852d9cf8fa599be274e25267"
server_key_iv: "e9bbc7b7ec30c2f681056f924e8474a0"
server_key_version: "encryptionkey@0"
I20260926 11:44:45.929342 28743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:45.955335 28743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:45.958087 28743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:45.958262 28743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:45.964787 28743 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:32771
I20260926 11:44:45.965054 28810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:32771 every 8 connection(s)
I20260926 11:44:45.965736 28743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:45.968557 28811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:45.972335 28811 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.62" port: 32771 } has no permanent_uuid. Determining permanent_uuid...
I20260926 11:44:45.974989 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 28743
I20260926 11:44:45.975124 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:45.975409 6528 external_mini_cluster.cc:1428] Setting key 71041341af07b6d2d073b1c85ec8784d
Sep 26 11:44:45 dist-test-slave-xxxx krb5kdc[28730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423085, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.96.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.61@KRBTEST.COM" created.
I20260926 11:44:45.987447 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:45.974723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53362 (local address 127.6.96.62:32771)
0926 11:44:45.975327 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:45.975344 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:45.975524 (+ 180us) server_negotiation.cc:407] Connection header received
0926 11:44:45.976784 (+ 1260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:45.976789 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:45.976975 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:45.977152 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:45.978012 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:45.978921 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:45.979873 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:45.980243 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:45.980387 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:45.982635 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:45.982640 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:45.982651 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:45.982672 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:45.984176 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:45.984646 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:45.984652 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:45.984654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:45.984759 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:45.985081 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:45.985084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:45.985086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:45.985547 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:45.985664 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:45.986117 (+ 453us) server_negotiation.cc:299] Negotiation successful
0926 11:44:45.986357 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"spinlock_wait_cycles":29952,"thread_start_us":156,"threads_started":1}
I20260926 11:44:45.987447 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:45.974723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:32771 (local address 127.0.0.1:53362)
0926 11:44:45.975338 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:45.975395 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:45.976462 (+ 1067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:45.977173 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:45.977178 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:45.977612 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:45.977866 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:45.977880 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:45.979031 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:45.979033 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:45.979762 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:45.979771 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:45.980386 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:45.980390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:45.980525 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:45.981061 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:45.981078 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:45.982484 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:45.984318 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:45.984326 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:45.984338 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:45.984547 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:45.984872 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:45.984875 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:45.984877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:45.984998 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:45.985678 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:45.985686 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:45.985853 (+ 167us) client_negotiation.cc:769] Sending connection context
0926 11:44:45.986055 (+ 202us) client_negotiation.cc:240] Negotiation successful
0926 11:44:45.986449 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":88,"thread_start_us":143,"threads_started":1}
I20260926 11:44:45.990067 28811 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 } has no permanent_uuid. Determining permanent_uuid...
I20260926 11:44:45.990845 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:45.990563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:38979 (local address 127.0.0.1:55904)
0926 11:44:45.990677 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:45.990758 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.61:38979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260926 11:44:45.991045 28761 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.96.61:38979: connect: Connection refused (error 111)
I20260926 11:44:45.991902 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:45.991703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:38979 (local address 127.0.0.1:55906)
0926 11:44:45.991802 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:45.991847 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.61:38979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260926 11:44:45.992046 28811 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.61:38979: Network error: Client connection negotiation failed: client connection to 127.6.96.61:38979: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.61.keytab.
Entry for principal kudu/127.6.96.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.61.keytab.
I20260926 11:44:46.002045 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.61:38979
--webserver_interface=127.6.96.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.61.keytab
--principal=kudu/127.6.96.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:33701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:32771,127.6.96.61:38979,127.6.96.60:44137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 11:44:46.041303 28811 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 } attempt: 1
I20260926 11:44:46.042176 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.041887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:38979 (local address 127.0.0.1:55916)
0926 11:44:46.042033 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:46.042097 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.61:38979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260926 11:44:46.042883 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.042641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:38979 (local address 127.0.0.1:55926)
0926 11:44:46.042748 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:46.042789 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.61:38979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260926 11:44:46.042990 28811 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.61:38979: Network error: Client connection negotiation failed: client connection to 127.6.96.61:38979: connect: Connection refused (error 111)
I20260926 11:44:46.111125 28811 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 } attempt: 2
I20260926 11:44:46.111861 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.111580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:38979 (local address 127.0.0.1:55936)
0926 11:44:46.111724 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:46.111786 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.61:38979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 11:44:46.112339 28818 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
I20260926 11:44:46.112543 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.112346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:38979 (local address 127.0.0.1:55942)
0926 11:44:46.112441 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:46.112484 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.61:38979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260926 11:44:46.112603 28818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:46.112648 28818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:46.112707 28818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:46.112727 28811 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.61:38979: Network error: Client connection negotiation failed: client connection to 127.6.96.61:38979: connect: Connection refused (error 111)
W20260926 11:44:46.116703 28818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:46.116755 28818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:46.116781 28818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:46.116804 28818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:46.116832 28818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:46.121978 28818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:33701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:32771,127.6.96.61:38979,127.6.96.60:44137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.61.keytab
--principal=kudu/127.6.96.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.61:38979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.96.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:46.123020 28818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:46.123994 28818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:46.131989 28824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:46.132035 28823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:46.132107 28826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:46.132387 28818 server_base.cc:1034] running on GCE node
I20260926 11:44:46.132946 28818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:46.133997 28818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:46.135146 28818 hybrid_clock.cc:648] HybridClock initialized: now 1790423086135124 us; error 35 us; skew 500 ppm
Sep 26 11:44:46 dist-test-slave-xxxx krb5kdc[28730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423086, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:46.137759 28818 init.cc:377] Logged in from keytab as kudu/127.6.96.61@KRBTEST.COM (short username kudu)
I20260926 11:44:46.138602 28818 webserver.cc:460] Webserver started at http://127.6.96.61:46129/ using document root <none> and password file <none>
I20260926 11:44:46.139160 28818 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:46.139245 28818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:46.139443 28818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:46.141419 28818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-1/data/instance:
uuid: "bb2b1cf8be844a03b584d50e49c34645"
format_stamp: "Formatted at 2026-09-26 11:44:46 on dist-test-slave-xxxx"
server_key: "4389dbb3431a50d560043cb9b5525c6e"
server_key_iv: "63cc85b2d3fd6ab8fce759872b8e5dee"
server_key_version: "encryptionkey@0"
I20260926 11:44:46.141944 28818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-1/wal/instance:
uuid: "bb2b1cf8be844a03b584d50e49c34645"
format_stamp: "Formatted at 2026-09-26 11:44:46 on dist-test-slave-xxxx"
server_key: "4389dbb3431a50d560043cb9b5525c6e"
server_key_iv: "63cc85b2d3fd6ab8fce759872b8e5dee"
server_key_version: "encryptionkey@0"
I20260926 11:44:46.145602 28818 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260926 11:44:46.147987 28833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:46.149191 28818 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 11:44:46.149304 28818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-1/wal
uuid: "bb2b1cf8be844a03b584d50e49c34645"
format_stamp: "Formatted at 2026-09-26 11:44:46 on dist-test-slave-xxxx"
server_key: "4389dbb3431a50d560043cb9b5525c6e"
server_key_iv: "63cc85b2d3fd6ab8fce759872b8e5dee"
server_key_version: "encryptionkey@0"
I20260926 11:44:46.149398 28818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:46.176028 28818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:46.179378 28818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:46.179559 28818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:46.186089 28818 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.61:38979
I20260926 11:44:46.186337 28885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.61:38979 every 8 connection(s)
I20260926 11:44:46.187187 28818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-1/data/info.pb
I20260926 11:44:46.187811 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 28818
I20260926 11:44:46.187932 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-1/wal/instance
I20260926 11:44:46.188211 6528 external_mini_cluster.cc:1428] Setting key 69a3f19969307aff4a2e16939f787644
I20260926 11:44:46.190495 28886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:46.194303 28886 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.62" port: 32771 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.96.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.60@KRBTEST.COM" created.
I20260926 11:44:46.203907 28811 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 } attempt: 3
Sep 26 11:44:46 dist-test-slave-xxxx krb5kdc[28730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423086, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.61@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:44:46.209578 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.196239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53372 (local address 127.6.96.62:32771)
0926 11:44:46.196372 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.196376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.197211 (+ 835us) server_negotiation.cc:407] Connection header received
0926 11:44:46.198330 (+ 1119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.198334 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.198396 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.198466 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.199691 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.200215 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.201794 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.202046 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.202139 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:46.205237 (+ 3098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:46.205241 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:46.205244 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:46.205271 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:46.207590 (+ 2319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.208070 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.208073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.208075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.208181 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.208485 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.208488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.208490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.208665 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:46.208750 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:46.209291 (+ 541us) server_negotiation.cc:299] Negotiation successful
0926 11:44:46.209412 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 11:44:46.210549 28889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.196600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:32771 (local address 127.0.0.1:53372)
0926 11:44:46.197067 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:46.197112 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:46.198105 (+ 993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:46.198647 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:46.198655 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:46.199035 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:46.199540 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:46.199554 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.200341 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.200344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:46.201629 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:46.201641 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.202216 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.202221 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:46.202436 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:46.203153 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:46.203175 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:46.205048 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:46.207693 (+ 2645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:46.207698 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:46.207711 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:46.207967 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:46.208262 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:46.208265 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:46.208267 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:46.208401 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:46.208760 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:46.208766 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:46.209049 (+ 283us) client_negotiation.cc:769] Sending connection context
0926 11:44:46.209227 (+ 178us) client_negotiation.cc:240] Negotiation successful
0926 11:44:46.209549 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":115,"threads_started":1}
Sep 26 11:44:46 dist-test-slave-xxxx krb5kdc[28730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423085, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for kudu/127.6.96.61@KRBTEST.COM
I20260926 11:44:46.212021 28886 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.60.keytab.
Entry for principal kudu/127.6.96.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.60.keytab.
I20260926 11:44:46.214452 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.60:44137
--webserver_interface=127.6.96.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.60.keytab
--principal=kudu/127.6.96.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:33701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:32771,127.6.96.61:38979,127.6.96.60:44137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 11:44:46.215181 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.204420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:38979 (local address 127.0.0.1:55946)
0926 11:44:46.204622 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:46.204635 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:46.204717 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:46.205406 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:46.205409 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:46.205593 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:46.205712 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:46.205720 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.207906 (+ 2186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.207910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:46.208628 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:46.208639 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.209270 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.209273 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:46.209406 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:46.209919 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:46.209933 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:46.211356 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:46.213543 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:46.213546 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:46.213549 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:46.213791 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:46.214078 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:46.214081 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:46.214083 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:46.214162 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:46.214661 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:46.214664 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:46.214773 (+ 109us) client_negotiation.cc:769] Sending connection context
0926 11:44:46.214885 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 11:44:46.215019 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260926 11:44:46.215926 28891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.204534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55946 (local address 127.6.96.61:38979)
0926 11:44:46.205005 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.205017 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.205042 (+ 25us) server_negotiation.cc:407] Connection header received
0926 11:44:46.205082 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.205088 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.205283 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.205399 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.206676 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.207801 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.208747 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.209154 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.209256 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:46.211487 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:46.211497 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:46.211531 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:46.211554 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:46.213422 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.213913 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.213917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.213918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.213987 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.214252 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.214255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.214257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.214539 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:46.215496 (+ 957us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:46.215597 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 11:44:46.215750 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":152,"threads_started":1}
Sep 26 11:44:46 dist-test-slave-xxxx krb5kdc[28730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423086, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.61@KRBTEST.COM for kudu/127.6.96.61@KRBTEST.COM
I20260926 11:44:46.217872 28811 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } has no permanent_uuid. Determining permanent_uuid...
I20260926 11:44:46.218564 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.218308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:44137 (local address 127.0.0.1:46960)
0926 11:44:46.218438 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:46.218493 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:44137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260926 11:44:46.219429 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.219197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:44137 (local address 127.0.0.1:46964)
0926 11:44:46.219327 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:46.219374 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:44137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260926 11:44:46.219571 28811 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.60:44137: Network error: Client connection negotiation failed: client connection to 127.6.96.60:44137: connect: Connection refused (error 111)
I20260926 11:44:46.221668 28889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.212555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:38979 (local address 127.0.0.1:55956)
0926 11:44:46.212665 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:46.212680 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:46.212750 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:46.213125 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:46.213128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:46.213335 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:46.213418 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:46.213423 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.214301 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.214304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:46.215062 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:46.215072 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.215597 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.215600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:46.215696 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:46.216058 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:46.216070 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:46.218150 (+ 2080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:46.220236 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:46.220240 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:46.220242 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:46.220482 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:46.220728 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:46.220731 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:46.220733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:46.220823 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:46.221138 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:46.221141 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:46.221252 (+ 111us) client_negotiation.cc:769] Sending connection context
0926 11:44:46.221361 (+ 109us) client_negotiation.cc:240] Negotiation successful
0926 11:44:46.221506 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260926 11:44:46.221698 28892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.212608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55956 (local address 127.6.96.61:38979)
0926 11:44:46.212953 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.212960 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.212969 (+ 9us) server_negotiation.cc:407] Connection header received
0926 11:44:46.212995 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.212997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.213027 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.213103 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.213511 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.214214 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.215169 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.215491 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.215603 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:46.218315 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:46.218319 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:46.218321 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:46.218346 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:46.220127 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.220584 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.220586 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.220587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.220653 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.220899 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.220900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.220901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.221056 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:46.221124 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:46.221393 (+ 269us) server_negotiation.cc:299] Negotiation successful
0926 11:44:46.221557 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":285,"thread_start_us":142,"threads_started":1}
I20260926 11:44:46.222570 28886 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } has no permanent_uuid. Determining permanent_uuid...
I20260926 11:44:46.223201 28889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.222997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:44137 (local address 127.0.0.1:46970)
0926 11:44:46.223087 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:46.223152 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:44137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260926 11:44:46.223407 28837 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.96.60:44137: connect: Connection refused (error 111)
I20260926 11:44:46.224227 28889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.224043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:44137 (local address 127.0.0.1:46974)
0926 11:44:46.224142 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:46.224182 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:44137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260926 11:44:46.224354 28886 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.60:44137: Network error: Client connection negotiation failed: client connection to 127.6.96.60:44137: connect: Connection refused (error 111)
I20260926 11:44:46.250739 28811 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } attempt: 1
I20260926 11:44:46.251617 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.251314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:44137 (local address 127.0.0.1:46984)
0926 11:44:46.251481 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:46.251543 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:44137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260926 11:44:46.252386 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.252195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:44137 (local address 127.0.0.1:46986)
0926 11:44:46.252280 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:46.252326 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:44137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 11:44:46.252508 28811 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.60:44137: Network error: Client connection negotiation failed: client connection to 127.6.96.60:44137: connect: Connection refused (error 111)
I20260926 11:44:46.273603 28886 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } attempt: 1
I20260926 11:44:46.274451 28889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.274157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:44137 (local address 127.0.0.1:46998)
0926 11:44:46.274312 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:46.274374 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:44137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260926 11:44:46.275166 28889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.274935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:44137 (local address 127.0.0.1:47014)
0926 11:44:46.275056 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:46.275104 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:44137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260926 11:44:46.275310 28886 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.60:44137: Network error: Client connection negotiation failed: client connection to 127.6.96.60:44137: connect: Connection refused (error 111)
W20260926 11:44:46.325959 28893 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 11:44:46.326231 28893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:46.326253 28893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:46.326305 28893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260926 11:44:46.327701 28811 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } attempt: 2
I20260926 11:44:46.328570 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.328240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:44137 (local address 127.0.0.1:47024)
0926 11:44:46.328399 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:46.328469 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:44137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260926 11:44:46.329304 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.329076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:44137 (local address 127.0.0.1:47040)
0926 11:44:46.329194 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:46.329243 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:44137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 11:44:46.329447 28811 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.60:44137: Network error: Client connection negotiation failed: client connection to 127.6.96.60:44137: connect: Connection refused (error 111)
W20260926 11:44:46.330669 28893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:46.330727 28893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:46.330755 28893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:46.330785 28893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:46.330813 28893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:46.336148 28893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:33701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:32771,127.6.96.61:38979,127.6.96.60:44137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.60.keytab
--principal=kudu/127.6.96.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.60:44137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.96.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:46.337204 28893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:46.338184 28893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 11:44:46.343448 28886 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } attempt: 2
I20260926 11:44:46.344215 28889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.343943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:44137 (local address 127.0.0.1:47046)
0926 11:44:46.344083 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:46.344143 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:44137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 11:44:46.344844 28889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.344646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:44137 (local address 127.0.0.1:47052)
0926 11:44:46.344745 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:46.344792 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:44137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260926 11:44:46.344947 28886 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.60:44137: Network error: Client connection negotiation failed: client connection to 127.6.96.60:44137: connect: Connection refused (error 111)
W20260926 11:44:46.346398 28900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:46.346391 28902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:46.346374 28899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:46.346771 28893 server_base.cc:1034] running on GCE node
I20260926 11:44:46.347349 28893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:46.348462 28893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:46.349615 28893 hybrid_clock.cc:648] HybridClock initialized: now 1790423086349605 us; error 37 us; skew 500 ppm
Sep 26 11:44:46 dist-test-slave-xxxx krb5kdc[28730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423086, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:46.352248 28893 init.cc:377] Logged in from keytab as kudu/127.6.96.60@KRBTEST.COM (short username kudu)
I20260926 11:44:46.353147 28893 webserver.cc:460] Webserver started at http://127.6.96.60:46559/ using document root <none> and password file <none>
I20260926 11:44:46.353701 28893 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:46.353761 28893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:46.353947 28893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:46.356045 28893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-2/data/instance:
uuid: "d71025f87f1646c481e5fb0df8f7bdbb"
format_stamp: "Formatted at 2026-09-26 11:44:46 on dist-test-slave-xxxx"
server_key: "bd9db08652780b38340e4746b05f7e43"
server_key_iv: "7c9e1a0f8399c7d0a2f4d72bcfb78d0f"
server_key_version: "encryptionkey@0"
I20260926 11:44:46.356585 28893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-2/wal/instance:
uuid: "d71025f87f1646c481e5fb0df8f7bdbb"
format_stamp: "Formatted at 2026-09-26 11:44:46 on dist-test-slave-xxxx"
server_key: "bd9db08652780b38340e4746b05f7e43"
server_key_iv: "7c9e1a0f8399c7d0a2f4d72bcfb78d0f"
server_key_version: "encryptionkey@0"
I20260926 11:44:46.360253 28893 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 11:44:46.362614 28909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:46.363901 28893 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:44:46.364003 28893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-2/wal
uuid: "d71025f87f1646c481e5fb0df8f7bdbb"
format_stamp: "Formatted at 2026-09-26 11:44:46 on dist-test-slave-xxxx"
server_key: "bd9db08652780b38340e4746b05f7e43"
server_key_iv: "7c9e1a0f8399c7d0a2f4d72bcfb78d0f"
server_key_version: "encryptionkey@0"
I20260926 11:44:46.364111 28893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:46.378206 28893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:46.381398 28893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:46.381572 28893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:46.388406 28893 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.60:44137
I20260926 11:44:46.388670 28961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.60:44137 every 8 connection(s)
I20260926 11:44:46.389402 28893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-2/data/info.pb
I20260926 11:44:46.391615 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 28893
I20260926 11:44:46.391729 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790422817500949-6528-0/minicluster-data/master-2/wal/instance
I20260926 11:44:46.391965 6528 external_mini_cluster.cc:1428] Setting key 97b79aac785221121e246d6c9a755469
I20260926 11:44:46.392303 28962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:46.397253 28962 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.62" port: 32771 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 11:44:46 dist-test-slave-xxxx krb5kdc[28730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.60@KRBTEST.COM
Sep 26 11:44:46 dist-test-slave-xxxx krb5kdc[28730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423086, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.60@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:44:46.406338 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.393214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47068 (local address 127.6.96.60:44137)
0926 11:44:46.393625 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.393643 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.393686 (+ 43us) server_negotiation.cc:407] Connection header received
0926 11:44:46.394468 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.394500 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.394958 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.395330 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.396034 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.397508 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.398401 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.399054 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.399256 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:46.401772 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:46.401778 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:46.401789 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:46.401821 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:46.403572 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.403983 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.403987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.403992 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.404103 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.404395 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.404397 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.404398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.404784 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:46.404964 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:46.405225 (+ 261us) server_negotiation.cc:299] Negotiation successful
0926 11:44:46.405495 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":112,"threads_started":1}
I20260926 11:44:46.407750 28966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.399123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:32771 (local address 127.0.0.1:53380)
0926 11:44:46.399604 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:46.399653 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:46.399772 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:46.400082 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:46.400087 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:46.400425 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:46.400554 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:46.400560 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.401326 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.401330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:46.402135 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:46.402145 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.402563 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.402566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:46.402676 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:46.403106 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:46.403117 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:46.404480 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:46.406093 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:46.406102 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:46.406108 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:46.406309 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:46.406577 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:46.406581 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:46.406582 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:46.406683 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:46.407189 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:46.407195 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:46.407309 (+ 114us) client_negotiation.cc:769] Sending connection context
0926 11:44:46.407449 (+ 140us) client_negotiation.cc:240] Negotiation successful
0926 11:44:46.407608 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":155,"threads_started":1}
I20260926 11:44:46.407807 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.399216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53380 (local address 127.6.96.62:32771)
0926 11:44:46.399368 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.399373 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.399739 (+ 366us) server_negotiation.cc:407] Connection header received
0926 11:44:46.399883 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.399887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.399945 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.400021 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.400680 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.401203 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.402241 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.402457 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.402550 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:46.404604 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:46.404608 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:46.404610 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:46.404634 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:46.405975 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.406398 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.406401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.406403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.406486 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.406804 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.406808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.406809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.407085 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:46.407180 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:46.407538 (+ 358us) server_negotiation.cc:299] Negotiation successful
0926 11:44:46.407658 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 11:44:46.408811 28962 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 11:44:46 dist-test-slave-xxxx krb5kdc[28730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423086, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.60@KRBTEST.COM for kudu/127.6.96.61@KRBTEST.COM
Sep 26 11:44:46 dist-test-slave-xxxx krb5kdc[28730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:44:46.416145 28966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.409248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:38979 (local address 127.0.0.1:55966)
0926 11:44:46.409347 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:46.409359 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:46.409423 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:46.409665 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:46.409668 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:46.409814 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:46.409896 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:46.409901 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.410601 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.410605 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:46.411162 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:46.411170 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.411593 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.411597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:46.411686 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:46.411994 (+ 308us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:46.412004 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:46.413118 (+ 1114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:46.414801 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:46.414804 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:46.414806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:46.415012 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:46.415262 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:46.415265 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:46.415267 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:46.415351 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:46.415717 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:46.415719 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:46.415812 (+ 93us) client_negotiation.cc:769] Sending connection context
0926 11:44:46.415918 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 11:44:46.416031 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260926 11:44:46.416287 28892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.409369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55966 (local address 127.6.96.61:38979)
0926 11:44:46.409488 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.409492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.409504 (+ 12us) server_negotiation.cc:407] Connection header received
0926 11:44:46.409541 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.409544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.409581 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.409641 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.409999 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.410497 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.411264 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.411502 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.411585 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:46.413245 (+ 1660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:46.413249 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:46.413252 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:46.413276 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:46.414717 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.415095 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.415099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.415101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.415184 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.415429 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.415432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.415433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.415625 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:46.415722 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:46.416012 (+ 290us) server_negotiation.cc:299] Negotiation successful
0926 11:44:46.416127 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 11:44:46.416875 28962 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } has no permanent_uuid. Determining permanent_uuid...
I20260926 11:44:46.417115 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.408754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53390 (local address 127.6.96.62:32771)
0926 11:44:46.408885 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.408888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.408899 (+ 11us) server_negotiation.cc:407] Connection header received
0926 11:44:46.409024 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.409027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.409067 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.409128 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.409762 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.410251 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.411044 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.411251 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.411332 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:46.413808 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:46.413811 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:46.413813 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:46.413833 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:46.415218 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.415609 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.415613 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.415614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.415692 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.415927 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.415930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.415932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.416146 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:46.416689 (+ 543us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:46.416871 (+ 182us) server_negotiation.cc:299] Negotiation successful
0926 11:44:46.416973 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Sep 26 11:44:46 dist-test-slave-xxxx krb5kdc[28730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423086, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.60@KRBTEST.COM for kudu/127.6.96.60@KRBTEST.COM
Sep 26 11:44:46 dist-test-slave-xxxx krb5kdc[28730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.61@KRBTEST.COM
I20260926 11:44:46.424839 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.417374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47074 (local address 127.6.96.60:44137)
0926 11:44:46.417470 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.417474 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.417536 (+ 62us) server_negotiation.cc:407] Connection header received
0926 11:44:46.417629 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.417632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.417664 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.417726 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.418047 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.418767 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.419786 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.420113 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.420222 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:46.421959 (+ 1737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:46.421963 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:46.421965 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:46.421986 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:46.423242 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.423653 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.423657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.423659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.423749 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.424028 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.424031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.424033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.424248 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:46.424326 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:46.424615 (+ 289us) server_negotiation.cc:299] Negotiation successful
0926 11:44:46.424716 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":37}
I20260926 11:44:46.424861 28966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.417355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:44137 (local address 127.0.0.1:47074)
0926 11:44:46.417485 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:46.417498 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:46.417570 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:46.417762 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:46.417765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:46.417897 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:46.417956 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:46.417960 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.418903 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.418906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:46.419684 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:46.419694 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.420218 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.420221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:46.420283 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:46.420670 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:46.420683 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:46.421810 (+ 1127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:46.423332 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:46.423335 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:46.423337 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:46.423552 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:46.423840 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:46.423844 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:46.423846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:46.423932 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:46.424336 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:46.424339 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:46.424435 (+ 96us) client_negotiation.cc:769] Sending connection context
0926 11:44:46.424544 (+ 109us) client_negotiation.cc:240] Negotiation successful
0926 11:44:46.424666 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260926 11:44:46.426406 28892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.418106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55982 (local address 127.6.96.61:38979)
0926 11:44:46.418203 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.418207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.418217 (+ 10us) server_negotiation.cc:407] Connection header received
0926 11:44:46.418313 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.418315 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.418352 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.418408 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.419063 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.419759 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.420596 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.420806 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.420888 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:46.423269 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:46.423273 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:46.423275 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:46.423299 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:46.424858 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.425334 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.425336 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.425337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.425410 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.425671 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.425674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.425675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.425871 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:46.426077 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:46.426210 (+ 133us) server_negotiation.cc:299] Negotiation successful
0926 11:44:46.426297 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 11:44:46.428609 28962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d71025f87f1646c481e5fb0df8f7bdbb: Bootstrap starting.
I20260926 11:44:46.428592 28811 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } attempt: 3
I20260926 11:44:46.429911 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 11:44:46.431434 28962 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d71025f87f1646c481e5fb0df8f7bdbb: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:46.431272 28785 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53390
I20260926 11:44:46.432338 28962 log.cc:824] T 00000000000000000000000000000000 P d71025f87f1646c481e5fb0df8f7bdbb: Log is configured to *not* fsync() on all Append() calls
Sep 26 11:44:46 dist-test-slave-xxxx krb5kdc[28730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423085, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for kudu/127.6.96.60@KRBTEST.COM
I20260926 11:44:46.434532 28962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d71025f87f1646c481e5fb0df8f7bdbb: No bootstrap required, opened a new log
I20260926 11:44:46.436086 28886 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } attempt: 3
I20260926 11:44:46.437464 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.428957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:44137 (local address 127.0.0.1:47086)
0926 11:44:46.429104 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:46.429118 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:46.429194 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:46.429555 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:46.429559 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:46.429738 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:46.429854 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:46.429861 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.430892 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.430896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:46.431656 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:46.431669 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.432283 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.432287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:46.432374 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:46.432865 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:46.432877 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:46.434171 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:46.435954 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:46.435957 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:46.435960 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:46.436213 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:46.436515 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:46.436519 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:46.436520 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:46.436611 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:46.436998 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:46.437001 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:46.437082 (+ 81us) client_negotiation.cc:769] Sending connection context
0926 11:44:46.437166 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 11:44:46.437302 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260926 11:44:46.437464 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.429235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47086 (local address 127.6.96.60:44137)
0926 11:44:46.429346 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.429350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.429362 (+ 12us) server_negotiation.cc:407] Connection header received
0926 11:44:46.429395 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.429397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.429438 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.429510 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.429963 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.430717 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.431797 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.432145 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.432595 (+ 450us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:46.434301 (+ 1706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:46.434305 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:46.434308 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:46.434331 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:46.435850 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.436326 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.436329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.436331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.436406 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.436726 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.436729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.436731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.436899 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:46.436980 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:46.437227 (+ 247us) server_negotiation.cc:299] Negotiation successful
0926 11:44:46.437326 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 11:44:46.438599 28962 raft_consensus.cc:348] T 00000000000000000000000000000000 P d71025f87f1646c481e5fb0df8f7bdbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 32771 } } peers { permanent_uuid: "bb2b1cf8be844a03b584d50e49c34645" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 } } peers { permanent_uuid: "d71025f87f1646c481e5fb0df8f7bdbb" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } }
I20260926 11:44:46.438870 28962 raft_consensus.cc:729] T 00000000000000000000000000000000 P d71025f87f1646c481e5fb0df8f7bdbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d71025f87f1646c481e5fb0df8f7bdbb, State: Initialized, Role: FOLLOWER
I20260926 11:44:46.439208 28962 consensus_queue.cc:260] T 00000000000000000000000000000000 P d71025f87f1646c481e5fb0df8f7bdbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 32771 } } peers { permanent_uuid: "bb2b1cf8be844a03b584d50e49c34645" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 } } peers { permanent_uuid: "d71025f87f1646c481e5fb0df8f7bdbb" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } }
I20260926 11:44:46.439669 28969 sys_catalog.cc:455] T 00000000000000000000000000000000 P d71025f87f1646c481e5fb0df8f7bdbb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 32771 } } peers { permanent_uuid: "bb2b1cf8be844a03b584d50e49c34645" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 } } peers { permanent_uuid: "d71025f87f1646c481e5fb0df8f7bdbb" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } } }
I20260926 11:44:46.439795 28969 sys_catalog.cc:458] T 00000000000000000000000000000000 P d71025f87f1646c481e5fb0df8f7bdbb [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:44:46.440495 28811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6: Bootstrap starting.
I20260926 11:44:46.440587 28962 sys_catalog.cc:564] T 00000000000000000000000000000000 P d71025f87f1646c481e5fb0df8f7bdbb [sys.catalog]: configured and running, proceeding with master startup.
Sep 26 11:44:46 dist-test-slave-xxxx krb5kdc[28730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423086, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.61@KRBTEST.COM for kudu/127.6.96.60@KRBTEST.COM
I20260926 11:44:46.444489 28811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:46.445149 28889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.436484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:44137 (local address 127.0.0.1:47102)
0926 11:44:46.436662 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:46.436676 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:46.436752 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:46.437865 (+ 1113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:46.437868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:46.438045 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:46.438151 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:46.438157 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.438879 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.438883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:46.439393 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:46.439401 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.439929 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.439933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:46.440025 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:46.440415 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:46.440427 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:46.441539 (+ 1112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:46.443538 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:46.443542 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:46.443544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:46.443798 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:46.444121 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:46.444124 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:46.444126 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:46.444218 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:46.444655 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:46.444657 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:46.444755 (+ 98us) client_negotiation.cc:769] Sending connection context
0926 11:44:46.444845 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 11:44:46.444981 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260926 11:44:46.445592 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.436535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47102 (local address 127.6.96.60:44137)
0926 11:44:46.437658 (+ 1123us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.437661 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.437672 (+ 11us) server_negotiation.cc:407] Connection header received
0926 11:44:46.437700 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.437703 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.437740 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.438239 (+ 499us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.438304 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.438756 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.439503 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.439788 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.439924 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:46.441686 (+ 1762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:46.441689 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:46.441691 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:46.441710 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:46.443406 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.443917 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.443921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.443922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.444018 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.444319 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.444322 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.444323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.444526 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:46.444611 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:46.445345 (+ 734us) server_negotiation.cc:299] Negotiation successful
0926 11:44:46.445451 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1070,"thread_start_us":110,"threads_started":1}
W20260926 11:44:46.446305 28980 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d71025f87f1646c481e5fb0df8f7bdbb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 11:44:46.446385 28980 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 11:44:46.446543 28811 log.cc:824] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:46.444283 28893 master_runner.cc:186] Error getting master registration for 127.6.96.62:32771: OK, instance_id { permanent_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" instance_seqno: 1790423085962905 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 11:44:46.447554 28893 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 11:44:46.448592 28886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb2b1cf8be844a03b584d50e49c34645: Bootstrap starting.
I20260926 11:44:46.449115 28811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6: No bootstrap required, opened a new log
I20260926 11:44:46.451293 28886 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb2b1cf8be844a03b584d50e49c34645: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:46.452173 28886 log.cc:824] T 00000000000000000000000000000000 P bb2b1cf8be844a03b584d50e49c34645: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:46.452358 28811 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 32771 } } peers { permanent_uuid: "bb2b1cf8be844a03b584d50e49c34645" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 } } peers { permanent_uuid: "d71025f87f1646c481e5fb0df8f7bdbb" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } }
I20260926 11:44:46.452602 28811 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4b87e2bdc624ead91584a0a7fc4ddd6, State: Initialized, Role: FOLLOWER
I20260926 11:44:46.452965 28811 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 32771 } } peers { permanent_uuid: "bb2b1cf8be844a03b584d50e49c34645" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 } } peers { permanent_uuid: "d71025f87f1646c481e5fb0df8f7bdbb" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } }
I20260926 11:44:46.453555 28984 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 32771 } } peers { permanent_uuid: "bb2b1cf8be844a03b584d50e49c34645" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 } } peers { permanent_uuid: "d71025f87f1646c481e5fb0df8f7bdbb" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } } }
I20260926 11:44:46.453796 28984 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:44:46.454349 28811 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:46.454402 28785 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 11:44:46.454448 28886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb2b1cf8be844a03b584d50e49c34645: No bootstrap required, opened a new log
I20260926 11:44:46.454512 28785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:46.455711 28785 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 32771 } } peers { permanent_uuid: "bb2b1cf8be844a03b584d50e49c34645" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 } } peers { permanent_uuid: "d71025f87f1646c481e5fb0df8f7bdbb" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } }
I20260926 11:44:46.456724 28785 leader_election.cc:288] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6 [CANDIDATE]: Term 1 election: Requested vote from peers bb2b1cf8be844a03b584d50e49c34645 (127.6.96.61:38979), d71025f87f1646c481e5fb0df8f7bdbb (127.6.96.60:44137)
I20260926 11:44:46.457469 28936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d71025f87f1646c481e5fb0df8f7bdbb"
I20260926 11:44:46.457726 28936 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d71025f87f1646c481e5fb0df8f7bdbb [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:46.459004 28936 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d71025f87f1646c481e5fb0df8f7bdbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4b87e2bdc624ead91584a0a7fc4ddd6 in term 1.
W20260926 11:44:46.460011 28764 tablet.cc:2369] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:44:46.459896 28886 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb2b1cf8be844a03b584d50e49c34645 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 32771 } } peers { permanent_uuid: "bb2b1cf8be844a03b584d50e49c34645" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 } } peers { permanent_uuid: "d71025f87f1646c481e5fb0df8f7bdbb" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } }
I20260926 11:44:46.460130 28886 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb2b1cf8be844a03b584d50e49c34645 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb2b1cf8be844a03b584d50e49c34645, State: Initialized, Role: FOLLOWER
I20260926 11:44:46.460446 28886 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb2b1cf8be844a03b584d50e49c34645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 32771 } } peers { permanent_uuid: "bb2b1cf8be844a03b584d50e49c34645" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 } } peers { permanent_uuid: "d71025f87f1646c481e5fb0df8f7bdbb" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } }
I20260926 11:44:46.461150 28762 leader_election.cc:302] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d71025f87f1646c481e5fb0df8f7bdbb, f4b87e2bdc624ead91584a0a7fc4ddd6; no voters:
I20260926 11:44:46.461524 28984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:46.461844 28886 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb2b1cf8be844a03b584d50e49c34645 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:46.461899 28860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bb2b1cf8be844a03b584d50e49c34645"
I20260926 11:44:46.462100 28860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb2b1cf8be844a03b584d50e49c34645 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:46.459542 28743 master_runner.cc:186] Error getting master registration for 127.6.96.61:38979: OK, instance_id { permanent_uuid: "bb2b1cf8be844a03b584d50e49c34645" instance_seqno: 1790423086184299 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 11:44:46.462749 28743 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 11:44:46.463444 28860 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bb2b1cf8be844a03b584d50e49c34645 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4b87e2bdc624ead91584a0a7fc4ddd6 in term 1.
I20260926 11:44:46.464187 28984 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6 [term 1 LEADER]: Becoming Leader. State: Replica: f4b87e2bdc624ead91584a0a7fc4ddd6, State: Running, Role: LEADER
I20260926 11:44:46.464617 28984 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 32771 } } peers { permanent_uuid: "bb2b1cf8be844a03b584d50e49c34645" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 } } peers { permanent_uuid: "d71025f87f1646c481e5fb0df8f7bdbb" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } }
W20260926 11:44:46.470299 28995 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 11:44:46.470425 28995 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 11:44:46.470652 29011 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4b87e2bdc624ead91584a0a7fc4ddd6. Latest consensus state: current_term: 1 leader_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 32771 } } peers { permanent_uuid: "bb2b1cf8be844a03b584d50e49c34645" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 } } peers { permanent_uuid: "d71025f87f1646c481e5fb0df8f7bdbb" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } } }
I20260926 11:44:46.472476 29011 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:46.472867 29018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:46.473781 29018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:46.478003 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.466717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47108 (local address 127.6.96.60:44137)
0926 11:44:46.466857 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.466861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.468587 (+ 1726us) server_negotiation.cc:407] Connection header received
0926 11:44:46.468785 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.468788 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.468838 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.468887 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.469888 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.470602 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.471713 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.472024 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.472098 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:46.474503 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:46.474507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:46.474509 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:46.474536 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:46.476218 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.476676 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.476680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.476682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.476773 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.477052 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.477056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.477057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.477261 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:46.477378 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:46.477728 (+ 350us) server_negotiation.cc:299] Negotiation successful
0926 11:44:46.477844 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 11:44:46.483063 29000 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb2b1cf8be844a03b584d50e49c34645 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 32771 } } peers { permanent_uuid: "bb2b1cf8be844a03b584d50e49c34645" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 } } peers { permanent_uuid: "d71025f87f1646c481e5fb0df8f7bdbb" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } } }
I20260926 11:44:46.483225 29000 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb2b1cf8be844a03b584d50e49c34645 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:44:46.484618 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.467465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53404 (local address 127.6.96.62:32771)
0926 11:44:46.467571 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.467576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.473352 (+ 5776us) server_negotiation.cc:407] Connection header received
0926 11:44:46.473530 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.473533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.473584 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.473634 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.474398 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.475192 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.476328 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.476633 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.476700 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:46.480953 (+ 4253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:46.480957 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:46.480960 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:46.480989 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:46.482707 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.483279 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.483282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.483284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.483376 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.483700 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.483704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.483706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.483908 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:46.484015 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:46.484331 (+ 316us) server_negotiation.cc:299] Negotiation successful
0926 11:44:46.484468 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 11:44:46.486881 28892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:46.467867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55984 (local address 127.6.96.61:38979)
0926 11:44:46.467980 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.467984 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.477287 (+ 9303us) server_negotiation.cc:407] Connection header received
0926 11:44:46.477460 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.477464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.477519 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.477570 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.478580 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.479327 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.480893 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:46.481249 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:46.481348 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:46.483203 (+ 1855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:46.483207 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:46.483210 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:46.483238 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:46.484876 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.485545 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.485549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.485550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.485628 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:46.485892 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:46.485896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:46.485897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:46.486115 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:46.486211 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:46.486553 (+ 342us) server_negotiation.cc:299] Negotiation successful
0926 11:44:46.486660 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260926 11:44:46.486691 29012 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb2b1cf8be844a03b584d50e49c34645: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260926 11:44:46.498921 28936 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d71025f87f1646c481e5fb0df8f7bdbb [term 1 FOLLOWER]: Refusing update from remote peer f4b87e2bdc624ead91584a0a7fc4ddd6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260926 11:44:46.498973 29012 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 11:44:46.499603 29011 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d71025f87f1646c481e5fb0df8f7bdbb" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:44:46.503482 28969 sys_catalog.cc:455] T 00000000000000000000000000000000 P d71025f87f1646c481e5fb0df8f7bdbb [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4b87e2bdc624ead91584a0a7fc4ddd6. Latest consensus state: current_term: 1 leader_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 32771 } } peers { permanent_uuid: "bb2b1cf8be844a03b584d50e49c34645" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 } } peers { permanent_uuid: "d71025f87f1646c481e5fb0df8f7bdbb" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } } }
I20260926 11:44:46.503615 28969 sys_catalog.cc:458] T 00000000000000000000000000000000 P d71025f87f1646c481e5fb0df8f7bdbb [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:44:46.504530 28860 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bb2b1cf8be844a03b584d50e49c34645 [term 1 FOLLOWER]: Refusing update from remote peer f4b87e2bdc624ead91584a0a7fc4ddd6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:44:46.505103 29011 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb2b1cf8be844a03b584d50e49c34645" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:44:46.508744 29000 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb2b1cf8be844a03b584d50e49c34645 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4b87e2bdc624ead91584a0a7fc4ddd6. Latest consensus state: current_term: 1 leader_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 32771 } } peers { permanent_uuid: "bb2b1cf8be844a03b584d50e49c34645" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 } } peers { permanent_uuid: "d71025f87f1646c481e5fb0df8f7bdbb" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } } }
I20260926 11:44:46.508865 29000 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb2b1cf8be844a03b584d50e49c34645 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:44:46.515269 29011 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 32771 } } peers { permanent_uuid: "bb2b1cf8be844a03b584d50e49c34645" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 } } peers { permanent_uuid: "d71025f87f1646c481e5fb0df8f7bdbb" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } } }
I20260926 11:44:46.515410 29011 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:46.516688 28969 sys_catalog.cc:455] T 00000000000000000000000000000000 P d71025f87f1646c481e5fb0df8f7bdbb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 32771 } } peers { permanent_uuid: "bb2b1cf8be844a03b584d50e49c34645" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 } } peers { permanent_uuid: "d71025f87f1646c481e5fb0df8f7bdbb" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } } }
I20260926 11:44:46.516814 28969 sys_catalog.cc:458] T 00000000000000000000000000000000 P d71025f87f1646c481e5fb0df8f7bdbb [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:44:46.517658 29021 mvcc.cc:205] Tried to move back new op lower bound from 7333572962290876416 to 7333572962162466816. Current Snapshot: MvccSnapshot[applied={T|T < 7333572962290876416}]
I20260926 11:44:46.518283 29019 mvcc.cc:205] Tried to move back new op lower bound from 7333572962290876416 to 7333572962162466816. Current Snapshot: MvccSnapshot[applied={T|T < 7333572962290876416}]
I20260926 11:44:46.520813 28984 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 32771 } } peers { permanent_uuid: "bb2b1cf8be844a03b584d50e49c34645" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 } } peers { permanent_uuid: "d71025f87f1646c481e5fb0df8f7bdbb" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } } }
I20260926 11:44:46.520943 28984 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:46.522123 29000 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb2b1cf8be844a03b584d50e49c34645 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b87e2bdc624ead91584a0a7fc4ddd6" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 32771 } } peers { permanent_uuid: "bb2b1cf8be844a03b584d50e49c34645" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 38979 } } peers { permanent_uuid: "d71025f87f1646c481e5fb0df8f7bdbb" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 44137 } } }
I20260926 11:44:46.522240 29000 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb2b1cf8be844a03b584d50e49c34645 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:44:46.522678 29023 mvcc.cc:205] Tried to move back new op lower bound from 7333572962290876416 to 7333572962162466816. Current Snapshot: MvccSnapshot[applied={T|T < 7333572962290876416}]
I20260926 11:44:46.528697 29018 catalog_manager.cc:1347] Generated new cluster ID: 034443ee4cfa4d84b99611dfe5e2e4ad
I20260926 11:44:46.528774 29018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:46.549578 29018 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:46.550977 29018 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:46.572644 29018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4b87e2bdc624ead91584a0a7fc4ddd6: Generated new TSK 0
I20260926 11:44:46.573637 29018 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 11:44:46.614007 28814 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:46.610718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53414 (local address 127.6.96.62:32771)
0926 11:44:46.610868 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.610873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.612744 (+ 1871us) server_negotiation.cc:407] Connection header received
0926 11:44:46.612931 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.612934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.612994 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.613044 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.613855 (+ 811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53414: BlockingRecv error: recv got EOF from 127.0.0.1:53414 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260926 11:44:46.615109 28965 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:46.611256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47122 (local address 127.6.96.60:44137)
0926 11:44:46.611367 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.611371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.613298 (+ 1927us) server_negotiation.cc:407] Connection header received
0926 11:44:46.614347 (+ 1049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.614350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.614398 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.614444 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.614985 (+ 541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47122: BlockingRecv error: recv got EOF from 127.0.0.1:47122 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260926 11:44:46.616618 28892 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:46.611962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55988 (local address 127.6.96.61:38979)
0926 11:44:46.612072 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.612076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.615566 (+ 3490us) server_negotiation.cc:407] Connection header received
0926 11:44:46.615912 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.615916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.615989 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.616040 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.616504 (+ 464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55988: BlockingRecv error: recv got EOF from 127.0.0.1:55988 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260926 11:44:46.621115 28892 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:46.619549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55992 (local address 127.6.96.61:38979)
0926 11:44:46.619651 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.619655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.620129 (+ 474us) server_negotiation.cc:407] Connection header received
0926 11:44:46.620360 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.620363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.620439 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.620496 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.621013 (+ 517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55992: BlockingRecv error: recv got EOF from 127.0.0.1:55992 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260926 11:44:46.625795 28892 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:46.624404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56006 (local address 127.6.96.61:38979)
0926 11:44:46.624499 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.624503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.624984 (+ 481us) server_negotiation.cc:407] Connection header received
0926 11:44:46.625149 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.625152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.625204 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.625250 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.625696 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56006: BlockingRecv error: recv got EOF from 127.0.0.1:56006 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260926 11:44:46.631234 28892 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:46.629746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56020 (local address 127.6.96.61:38979)
0926 11:44:46.629847 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.629851 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.630368 (+ 517us) server_negotiation.cc:407] Connection header received
0926 11:44:46.630541 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.630544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.630592 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.630639 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.631139 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56020: BlockingRecv error: recv got EOF from 127.0.0.1:56020 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260926 11:44:46.638516 28892 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:46.637316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56032 (local address 127.6.96.61:38979)
0926 11:44:46.637458 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.637463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.637672 (+ 209us) server_negotiation.cc:407] Connection header received
0926 11:44:46.637794 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.637798 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.637850 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.637939 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.638393 (+ 454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56032: BlockingRecv error: recv got EOF from 127.0.0.1:56032 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260926 11:44:46.649878 28892 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:46.648526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56036 (local address 127.6.96.61:38979)
0926 11:44:46.648653 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.648658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.648941 (+ 283us) server_negotiation.cc:407] Connection header received
0926 11:44:46.649089 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.649093 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.649155 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.649249 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.649754 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56036: BlockingRecv error: recv got EOF from 127.0.0.1:56036 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260926 11:44:46.669351 28892 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:46.667955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56038 (local address 127.6.96.61:38979)
0926 11:44:46.668084 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.668088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.668341 (+ 253us) server_negotiation.cc:407] Connection header received
0926 11:44:46.668570 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.668575 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.668646 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.668746 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.669226 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56038: BlockingRecv error: recv got EOF from 127.0.0.1:56038 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260926 11:44:46.704506 28892 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:46.703515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56042 (local address 127.6.96.61:38979)
0926 11:44:46.703671 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.703676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.703791 (+ 115us) server_negotiation.cc:407] Connection header received
0926 11:44:46.703906 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.703909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.703957 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.704038 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.704407 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56042: BlockingRecv error: recv got EOF from 127.0.0.1:56042 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260926 11:44:46.771806 28892 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:46.770690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56050 (local address 127.6.96.61:38979)
0926 11:44:46.770866 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.770871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.770970 (+ 99us) server_negotiation.cc:407] Connection header received
0926 11:44:46.771127 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.771131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.771179 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.771246 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.771685 (+ 439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56050: BlockingRecv error: recv got EOF from 127.0.0.1:56050 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260926 11:44:46.903065 28892 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:46.901946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56060 (local address 127.6.96.61:38979)
0926 11:44:46.902119 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:46.902123 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:46.902215 (+ 92us) server_negotiation.cc:407] Connection header received
0926 11:44:46.902365 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:46.902368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:46.902418 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:46.902488 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:46.902941 (+ 453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56060: BlockingRecv error: recv got EOF from 127.0.0.1:56060 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260926 11:44:47.162225 28892 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:47.161215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56074 (local address 127.6.96.61:38979)
0926 11:44:47.161349 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:47.161353 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:47.161371 (+ 18us) server_negotiation.cc:407] Connection header received
0926 11:44:47.161533 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:47.161536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:47.161581 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:47.161648 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:47.162107 (+ 459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56074: BlockingRecv error: recv got EOF from 127.0.0.1:56074 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 11:44:47.447686 28980 catalog_manager.cc:1259] Loaded cluster ID: 034443ee4cfa4d84b99611dfe5e2e4ad
I20260926 11:44:47.447772 28980 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d71025f87f1646c481e5fb0df8f7bdbb: loading cluster ID for follower catalog manager: success
I20260926 11:44:47.449425 28980 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d71025f87f1646c481e5fb0df8f7bdbb: acquiring CA information for follower catalog manager: success
I20260926 11:44:47.450559 28980 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d71025f87f1646c481e5fb0df8f7bdbb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 11:44:47.500260 29012 catalog_manager.cc:1259] Loaded cluster ID: 034443ee4cfa4d84b99611dfe5e2e4ad
I20260926 11:44:47.500349 29012 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bb2b1cf8be844a03b584d50e49c34645: loading cluster ID for follower catalog manager: success
I20260926 11:44:47.501993 29012 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bb2b1cf8be844a03b584d50e49c34645: acquiring CA information for follower catalog manager: success
I20260926 11:44:47.503197 29012 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bb2b1cf8be844a03b584d50e49c34645: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 11:44:47.680362 29093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:47.676389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56086 (local address 127.6.96.61:38979)
0926 11:44:47.676640 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:47.676645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:47.676715 (+ 70us) server_negotiation.cc:407] Connection header received
0926 11:44:47.676885 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:47.676888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:47.677012 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:47.677145 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:44:47.677420 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:47.677956 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:47.678854 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:47.679096 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:47.679188 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:47.679518 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:44:47.679963 (+ 445us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:44:47.680077 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:47.680141 (+ 64us) server_negotiation.cc:299] Negotiation successful
0926 11:44:47.680204 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":65,"threads_started":1}
I20260926 11:44:47.686990 29099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:47.683617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47126 (local address 127.6.96.60:44137)
0926 11:44:47.683820 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:47.683823 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:47.683836 (+ 13us) server_negotiation.cc:407] Connection header received
0926 11:44:47.683929 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:47.683931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:47.684053 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:47.684153 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:44:47.684331 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:47.684836 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:47.685622 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:47.685854 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:47.685946 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:47.686176 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:44:47.686600 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:44:47.686691 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:47.686780 (+ 89us) server_negotiation.cc:299] Negotiation successful
0926 11:44:47.686853 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":69,"threads_started":1}
I20260926 11:44:47.693646 29093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:47.690280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56094 (local address 127.6.96.61:38979)
0926 11:44:47.690420 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:47.690424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:47.690437 (+ 13us) server_negotiation.cc:407] Connection header received
0926 11:44:47.690485 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:47.690488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:47.690534 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:47.690592 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:44:47.690802 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:47.691466 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:47.692518 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:47.692802 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:47.692890 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:47.693175 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:44:47.693256 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:44:47.693298 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:47.693458 (+ 160us) server_negotiation.cc:299] Negotiation successful
0926 11:44:47.693513 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 11:44:47.694067 29099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:47.689880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47134 (local address 127.6.96.60:44137)
0926 11:44:47.689992 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:47.689997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:47.690357 (+ 360us) server_negotiation.cc:407] Connection header received
0926 11:44:47.690962 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:47.690966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:47.691018 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:47.691062 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:44:47.691384 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:47.692001 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:47.693031 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:47.693320 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:47.693413 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:47.693623 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:44:47.693694 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:44:47.693756 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:47.693897 (+ 141us) server_negotiation.cc:299] Negotiation successful
0926 11:44:47.693945 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 11:44:47.694301 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:47.689782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53420 (local address 127.6.96.62:32771)
0926 11:44:47.690134 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:47.690137 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:47.690150 (+ 13us) server_negotiation.cc:407] Connection header received
0926 11:44:47.690476 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:47.690479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:47.690597 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:47.690703 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:44:47.691021 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:47.691727 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:47.692694 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:47.693029 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:47.693113 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:47.693400 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:44:47.693744 (+ 344us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:44:47.694067 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:47.694103 (+ 36us) server_negotiation.cc:299] Negotiation successful
0926 11:44:47.694160 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":84,"threads_started":1}
I20260926 11:44:47.697551 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 28743
I20260926 11:44:47.703188 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 28818
I20260926 11:44:47.708027 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 28893
2026-09-26T11:44:47Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3023 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 11:44:47 dist-test-slave-xxxx krb5kdc[29110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 11:44:47 dist-test-slave-xxxx krb5kdc[29110](info): set up 2 sockets
Sep 26 11:44:47 dist-test-slave-xxxx krb5kdc[29110](info): commencing operation
krb5kdc: starting...
W20260926 11:44:48.666962 6528 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/krb5kdc/test-user.keytab.
Sep 26 11:44:48 dist-test-slave-xxxx krb5kdc[29110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T11:44:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T11:44:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.96.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
Entry for principal kudu/127.6.96.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab.
I20260926 11:44:48.785889 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:40039
--webserver_interface=127.6.96.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:43099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:40039,127.6.96.61:39523,127.6.96.60:39353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:48.895263 29123 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 11:44:48.895534 29123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:48.895560 29123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:48.895617 29123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:48.899609 29123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:48.899669 29123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:48.899688 29123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:48.899703 29123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:48.899722 29123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:48.904992 29123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:43099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:40039,127.6.96.61:39523,127.6.96.60:39353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.62.keytab
--principal=kudu/127.6.96.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.62:40039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.96.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:48.906059 29123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:48.907104 29123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:48.915130 29131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:48.915182 29129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:48.915170 29128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:48.915249 29123 server_base.cc:1034] running on GCE node
I20260926 11:44:48.915941 29123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:48.916997 29123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:48.918179 29123 hybrid_clock.cc:648] HybridClock initialized: now 1790423088918161 us; error 28 us; skew 500 ppm
Sep 26 11:44:48 dist-test-slave-xxxx krb5kdc[29110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423088, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:48.920815 29123 init.cc:377] Logged in from keytab as kudu/127.6.96.62@KRBTEST.COM (short username kudu)
I20260926 11:44:48.921696 29123 webserver.cc:460] Webserver started at http://127.6.96.62:36545/ using document root <none> and password file <none>
I20260926 11:44:48.922214 29123 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:48.922277 29123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:48.922468 29123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:48.924484 29123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-0/data/instance:
uuid: "c5c5b68dd9d447e89dcbfefeaa845525"
format_stamp: "Formatted at 2026-09-26 11:44:48 on dist-test-slave-xxxx"
server_key: "23b1b69198ef444acea27f93e21654b8"
server_key_iv: "77afde929d5852b767c5f809e0ec6ca5"
server_key_version: "encryptionkey@0"
I20260926 11:44:48.925015 29123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-0/wal/instance:
uuid: "c5c5b68dd9d447e89dcbfefeaa845525"
format_stamp: "Formatted at 2026-09-26 11:44:48 on dist-test-slave-xxxx"
server_key: "23b1b69198ef444acea27f93e21654b8"
server_key_iv: "77afde929d5852b767c5f809e0ec6ca5"
server_key_version: "encryptionkey@0"
I20260926 11:44:48.928722 29123 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260926 11:44:48.931135 29138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:48.932394 29123 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 11:44:48.932510 29123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-0/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-0/wal
uuid: "c5c5b68dd9d447e89dcbfefeaa845525"
format_stamp: "Formatted at 2026-09-26 11:44:48 on dist-test-slave-xxxx"
server_key: "23b1b69198ef444acea27f93e21654b8"
server_key_iv: "77afde929d5852b767c5f809e0ec6ca5"
server_key_version: "encryptionkey@0"
I20260926 11:44:48.932621 29123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:48.950176 29123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:48.953131 29123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:48.953316 29123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:48.960228 29123 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.62:40039
I20260926 11:44:48.960496 29190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.62:40039 every 8 connection(s)
I20260926 11:44:48.961217 29123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-0/data/info.pb
I20260926 11:44:48.961607 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 29123
I20260926 11:44:48.961726 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-0/wal/instance
I20260926 11:44:48.962028 6528 external_mini_cluster.cc:1428] Setting key 099b9cbbb2c56e60e48855b9c83c7e92
I20260926 11:44:48.964568 29191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:48.969938 29191 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40039 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.96.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.61@KRBTEST.COM" created.
Sep 26 11:44:48 dist-test-slave-xxxx krb5kdc[29110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423088, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:44:48.984843 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:48.972615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:40039 (local address 127.0.0.1:44432)
0926 11:44:48.973192 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:48.973247 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:48.974205 (+ 958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:48.974964 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:48.974970 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:48.975401 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:48.975661 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:48.975676 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:48.976802 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:48.976805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:48.977553 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:48.977561 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:48.978168 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:48.978172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:48.978294 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:48.978958 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:48.978974 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:48.980338 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:48.981995 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:48.982004 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:48.982014 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:48.982222 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:48.982510 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:48.982513 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:48.982515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:48.982618 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:48.983202 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:48.983208 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:48.983335 (+ 127us) client_negotiation.cc:769] Sending connection context
0926 11:44:48.983487 (+ 152us) client_negotiation.cc:240] Negotiation successful
0926 11:44:48.983860 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":111,"spinlock_wait_cycles":65280,"thread_start_us":212,"threads_started":1}
I20260926 11:44:48.984843 29194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:48.972615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44432 (local address 127.6.96.62:40039)
0926 11:44:48.973110 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:48.973129 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:48.973364 (+ 235us) server_negotiation.cc:407] Connection header received
0926 11:44:48.974529 (+ 1165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:48.974533 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:48.974740 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:48.974930 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:48.975808 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:48.976676 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:48.977664 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:48.978025 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:48.978168 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:48.980471 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:48.980476 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:48.980488 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:48.980509 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:48.981902 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:48.982319 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:48.982325 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:48.982327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:48.982431 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:48.982698 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:48.982701 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:48.982703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:48.983100 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:48.983217 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:48.983557 (+ 340us) server_negotiation.cc:299] Negotiation successful
0926 11:44:48.983803 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":350,"thread_start_us":142,"threads_started":1}
I20260926 11:44:48.987425 29191 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 } has no permanent_uuid. Determining permanent_uuid...
I20260926 11:44:48.988132 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:48.987867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:39523 (local address 127.0.0.1:58822)
0926 11:44:48.987982 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:48.988064 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.61:39523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260926 11:44:48.988317 29139 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.96.61:39523: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.61.keytab.
Entry for principal kudu/127.6.96.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.61.keytab.
I20260926 11:44:48.989192 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:48.988965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:39523 (local address 127.0.0.1:58836)
0926 11:44:48.989069 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:48.989123 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.61:39523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260926 11:44:48.989789 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.61:39523
--webserver_interface=127.6.96.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.61.keytab
--principal=kudu/127.6.96.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:43099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:40039,127.6.96.61:39523,127.6.96.60:39353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 11:44:48.990942 29191 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.61:39523: Network error: Client connection negotiation failed: client connection to 127.6.96.61:39523: connect: Connection refused (error 111)
I20260926 11:44:49.040225 29191 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 } attempt: 1
I20260926 11:44:49.041214 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.040754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:39523 (local address 127.0.0.1:58846)
0926 11:44:49.041032 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:49.041113 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.61:39523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 11:44:49.042126 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.041865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:39523 (local address 127.0.0.1:58858)
0926 11:44:49.042000 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:49.042055 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.61:39523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260926 11:44:49.042287 29191 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.61:39523: Network error: Client connection negotiation failed: client connection to 127.6.96.61:39523: connect: Connection refused (error 111)
W20260926 11:44:49.099951 29198 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 11:44:49.100232 29198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:49.100255 29198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:49.100301 29198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:49.104286 29198 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:49.104342 29198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:49.104362 29198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:49.104375 29198 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:49.104393 29198 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:49.109627 29198 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:43099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:40039,127.6.96.61:39523,127.6.96.60:39353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.61.keytab
--principal=kudu/127.6.96.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.61:39523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.96.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:49.110441 29191 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 } attempt: 2
I20260926 11:44:49.110788 29198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:49.111325 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.110990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:39523 (local address 127.0.0.1:58866)
0926 11:44:49.111166 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:49.111238 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.61:39523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260926 11:44:49.112003 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.111811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:39523 (local address 127.0.0.1:58870)
0926 11:44:49.111907 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:49.111953 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.61:39523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260926 11:44:49.112104 29198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 11:44:49.112171 29191 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.61:39523: Network error: Client connection negotiation failed: client connection to 127.6.96.61:39523: connect: Connection refused (error 111)
W20260926 11:44:49.120170 29204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:49.120172 29206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:49.120172 29203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:49.120380 29198 server_base.cc:1034] running on GCE node
I20260926 11:44:49.120975 29198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:49.122031 29198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:49.123209 29198 hybrid_clock.cc:648] HybridClock initialized: now 1790423089123192 us; error 29 us; skew 500 ppm
Sep 26 11:44:49 dist-test-slave-xxxx krb5kdc[29110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423089, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:49.125721 29198 init.cc:377] Logged in from keytab as kudu/127.6.96.61@KRBTEST.COM (short username kudu)
I20260926 11:44:49.126629 29198 webserver.cc:460] Webserver started at http://127.6.96.61:45653/ using document root <none> and password file <none>
I20260926 11:44:49.127189 29198 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:49.127254 29198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:49.127444 29198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:49.129395 29198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-1/data/instance:
uuid: "099195bf4c0b4fdc9f85853c87a165c1"
format_stamp: "Formatted at 2026-09-26 11:44:49 on dist-test-slave-xxxx"
server_key: "228ca020aafb1e53b8e9af5bdf4e8013"
server_key_iv: "754fbee395337566c747d1ff2e88b5d1"
server_key_version: "encryptionkey@0"
I20260926 11:44:49.129920 29198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-1/wal/instance:
uuid: "099195bf4c0b4fdc9f85853c87a165c1"
format_stamp: "Formatted at 2026-09-26 11:44:49 on dist-test-slave-xxxx"
server_key: "228ca020aafb1e53b8e9af5bdf4e8013"
server_key_iv: "754fbee395337566c747d1ff2e88b5d1"
server_key_version: "encryptionkey@0"
I20260926 11:44:49.133476 29198 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260926 11:44:49.135825 29213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:49.137025 29198 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:44:49.137140 29198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-1/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-1/wal
uuid: "099195bf4c0b4fdc9f85853c87a165c1"
format_stamp: "Formatted at 2026-09-26 11:44:49 on dist-test-slave-xxxx"
server_key: "228ca020aafb1e53b8e9af5bdf4e8013"
server_key_iv: "754fbee395337566c747d1ff2e88b5d1"
server_key_version: "encryptionkey@0"
I20260926 11:44:49.137228 29198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:49.151386 29198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:49.154587 29198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:49.154770 29198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:49.161500 29198 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.61:39523
I20260926 11:44:49.161813 29265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.61:39523 every 8 connection(s)
I20260926 11:44:49.162514 29198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-1/data/info.pb
I20260926 11:44:49.165275 29266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:49.165566 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 29198
I20260926 11:44:49.165678 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-1/wal/instance
I20260926 11:44:49.166013 6528 external_mini_cluster.cc:1428] Setting key 08a68a0a80d1347992c38571f564aa39
I20260926 11:44:49.169562 29266 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40039 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.96.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.96.60@KRBTEST.COM" created.
Sep 26 11:44:49 dist-test-slave-xxxx krb5kdc[29110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423089, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.61@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:44:49.183656 29194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.171401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44434 (local address 127.6.96.62:40039)
0926 11:44:49.171562 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.171567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.172388 (+ 821us) server_negotiation.cc:407] Connection header received
0926 11:44:49.173329 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.173332 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.173384 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.173467 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.174652 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.175250 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.176781 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.177150 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.177271 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:49.179623 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:49.179627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:49.179630 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:49.179661 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:49.181201 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.181665 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.181669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.181671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.181754 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.182049 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.182052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.182055 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.182288 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:49.182425 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:49.183382 (+ 957us) server_negotiation.cc:299] Negotiation successful
0926 11:44:49.183491 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 11:44:49.183923 29269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.171768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:40039 (local address 127.0.0.1:44434)
0926 11:44:49.172219 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:49.172269 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:49.173121 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:49.173633 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:49.173640 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:49.173963 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:49.174431 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:49.174466 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.175383 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.175387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:49.176643 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:49.176653 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.177300 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.177302 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:49.177440 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:49.178012 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:49.178029 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:49.179450 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:49.181316 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:49.181321 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:49.181333 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:49.181575 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:49.181840 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:49.181842 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:49.181844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:49.181966 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:49.182381 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:49.182386 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:49.182620 (+ 234us) client_negotiation.cc:769] Sending connection context
0926 11:44:49.182767 (+ 147us) client_negotiation.cc:240] Negotiation successful
0926 11:44:49.183070 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":106,"threads_started":1}
I20260926 11:44:49.185302 29266 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.96.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.60.keytab.
Entry for principal kudu/127.6.96.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.60.keytab.
I20260926 11:44:49.190479 6528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBunNuW/build/debug/bin/kudu
/tmp/dist-test-taskBunNuW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.60:39353
--webserver_interface=127.6.96.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.60.keytab
--principal=kudu/127.6.96.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.96.20:43099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.96.62:40039,127.6.96.61:39523,127.6.96.60:39353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 11:44:49 dist-test-slave-xxxx krb5kdc[29110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423089, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.61@KRBTEST.COM for kudu/127.6.96.61@KRBTEST.COM
I20260926 11:44:49.194406 29269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.185786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:39523 (local address 127.0.0.1:58880)
0926 11:44:49.185921 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:49.185973 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:49.186052 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:49.186546 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:49.186550 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:49.186731 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:49.186867 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:49.186875 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.187831 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.187834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:49.188446 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:49.188455 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.189004 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.189007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:49.189083 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:49.189510 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:49.189524 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:49.190925 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:49.192909 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:49.192912 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:49.192914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:49.193159 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:49.193398 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:49.193401 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:49.193402 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:49.193484 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:49.193893 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:49.193896 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:49.193987 (+ 91us) client_negotiation.cc:769] Sending connection context
0926 11:44:49.194095 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 11:44:49.194226 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260926 11:44:49.194453 29271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.185900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58880 (local address 127.6.96.61:39523)
0926 11:44:49.186184 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.186194 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.186218 (+ 24us) server_negotiation.cc:407] Connection header received
0926 11:44:49.186264 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.186267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.186417 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.186519 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.186973 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.187725 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.188547 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.188909 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.188989 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:49.191043 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:49.191054 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:49.191069 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:49.191090 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:49.192802 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.193247 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.193252 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.193253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.193325 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.193565 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.193567 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.193568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.193804 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:49.193902 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:49.194197 (+ 295us) server_negotiation.cc:299] Negotiation successful
0926 11:44:49.194314 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":125,"threads_started":1}
I20260926 11:44:49.196192 29266 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } has no permanent_uuid. Determining permanent_uuid...
I20260926 11:44:49.196888 29269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.196651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:39353 (local address 127.0.0.1:34712)
0926 11:44:49.196761 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:49.196828 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:39353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260926 11:44:49.197089 29217 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.96.60:39353: connect: Connection refused (error 111)
I20260926 11:44:49.197876 29269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.197688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:39353 (local address 127.0.0.1:34722)
0926 11:44:49.197788 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:49.197826 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:39353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260926 11:44:49.197991 29266 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.60:39353: Network error: Client connection negotiation failed: client connection to 127.6.96.60:39353: connect: Connection refused (error 111)
I20260926 11:44:49.203320 29191 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 } attempt: 3
Sep 26 11:44:49 dist-test-slave-xxxx krb5kdc[29110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423088, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for kudu/127.6.96.61@KRBTEST.COM
I20260926 11:44:49.211584 29271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.203840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58890 (local address 127.6.96.61:39523)
0926 11:44:49.203943 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.203946 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.203960 (+ 14us) server_negotiation.cc:407] Connection header received
0926 11:44:49.204103 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.204105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.204142 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.204228 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.204665 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.205183 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.206292 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.206509 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.206576 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:49.208583 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:49.208586 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:49.208588 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:49.208609 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:49.210042 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.210471 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.210473 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.210475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.210541 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.210774 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.210776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.210777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.211008 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:49.211084 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:49.211359 (+ 275us) server_negotiation.cc:299] Negotiation successful
0926 11:44:49.211451 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 11:44:49.211603 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.203769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:39523 (local address 127.0.0.1:58890)
0926 11:44:49.203904 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:49.203919 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:49.204007 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:49.204240 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:49.204243 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:49.204432 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:49.204559 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:49.204567 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.205296 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.205300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:49.206198 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:49.206209 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.206592 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.206595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:49.206702 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:49.207128 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:49.207142 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:49.208455 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:49.210145 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:49.210148 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:49.210150 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:49.210384 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:49.210614 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:49.210616 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:49.210618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:49.210704 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:49.211094 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:49.211098 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:49.211208 (+ 110us) client_negotiation.cc:769] Sending connection context
0926 11:44:49.211318 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 11:44:49.211448 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260926 11:44:49.212795 29191 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } has no permanent_uuid. Determining permanent_uuid...
I20260926 11:44:49.213498 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.213262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:39353 (local address 127.0.0.1:34730)
0926 11:44:49.213384 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:49.213434 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:39353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 11:44:49.214154 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.213900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:39353 (local address 127.0.0.1:34732)
0926 11:44:49.214055 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:49.214097 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:39353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260926 11:44:49.214289 29191 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.60:39353: Network error: Client connection negotiation failed: client connection to 127.6.96.60:39353: connect: Connection refused (error 111)
I20260926 11:44:49.245469 29191 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } attempt: 1
I20260926 11:44:49.246325 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.246037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:39353 (local address 127.0.0.1:34740)
0926 11:44:49.246186 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:49.246248 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:39353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260926 11:44:49.247113 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.246897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:39353 (local address 127.0.0.1:34746)
0926 11:44:49.246996 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:49.247049 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:39353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 11:44:49.247277 29191 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.60:39353: Network error: Client connection negotiation failed: client connection to 127.6.96.60:39353: connect: Connection refused (error 111)
I20260926 11:44:49.247228 29266 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } attempt: 1
I20260926 11:44:49.247896 29269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.247639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:39353 (local address 127.0.0.1:34752)
0926 11:44:49.247755 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:49.247805 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:39353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260926 11:44:49.248574 29269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.248380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:39353 (local address 127.0.0.1:34768)
0926 11:44:49.248477 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:49.248520 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:39353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260926 11:44:49.248714 29266 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.60:39353: Network error: Client connection negotiation failed: client connection to 127.6.96.60:39353: connect: Connection refused (error 111)
W20260926 11:44:49.304488 29272 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 11:44:49.304780 29272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 11:44:49.304802 29272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 11:44:49.304840 29272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 11:44:49.309131 29272 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 11:44:49.309202 29272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 11:44:49.309221 29272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 11:44:49.309237 29272 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 11:44:49.309252 29272 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 11:44:49.314664 29272 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.96.20:43099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.96.62:40039,127.6.96.61:39523,127.6.96.60:39353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/krb5kdc/kudu_127.6.96.60.keytab
--principal=kudu/127.6.96.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.96.60:39353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.96.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 10:47:36 UTC on ccycloud.quasar-emzlrj.root.comops.site
build id 4673837
I20260926 11:44:49.315773 29272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 11:44:49.316910 29272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 11:44:49.316880 29266 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } attempt: 2
I20260926 11:44:49.317602 29269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.317328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:39353 (local address 127.0.0.1:34774)
0926 11:44:49.317473 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:49.317528 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:39353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260926 11:44:49.318272 29269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.318088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:39353 (local address 127.0.0.1:34788)
0926 11:44:49.318177 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:49.318220 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:39353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260926 11:44:49.318383 29266 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.60:39353: Network error: Client connection negotiation failed: client connection to 127.6.96.60:39353: connect: Connection refused (error 111)
I20260926 11:44:49.322427 29191 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } attempt: 2
I20260926 11:44:49.323069 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.322813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:39353 (local address 127.0.0.1:34798)
0926 11:44:49.322930 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:49.322995 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:39353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260926 11:44:49.323698 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.323531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:39353 (local address 127.0.0.1:34800)
0926 11:44:49.323611 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:49.323651 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.96.60:39353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260926 11:44:49.323817 29191 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.96.60:39353: Network error: Client connection negotiation failed: client connection to 127.6.96.60:39353: connect: Connection refused (error 111)
W20260926 11:44:49.325686 29278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 11:44:49.325724 29281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:49.325905 29272 server_base.cc:1034] running on GCE node
W20260926 11:44:49.325692 29279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 11:44:49.326648 29272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 11:44:49.327764 29272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 11:44:49.328919 29272 hybrid_clock.cc:648] HybridClock initialized: now 1790423089328903 us; error 54 us; skew 500 ppm
Sep 26 11:44:49 dist-test-slave-xxxx krb5kdc[29110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423089, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 11:44:49.331557 29272 init.cc:377] Logged in from keytab as kudu/127.6.96.60@KRBTEST.COM (short username kudu)
I20260926 11:44:49.332427 29272 webserver.cc:460] Webserver started at http://127.6.96.60:42749/ using document root <none> and password file <none>
I20260926 11:44:49.332957 29272 fs_manager.cc:359] Metadata directory not provided
I20260926 11:44:49.333024 29272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 11:44:49.333235 29272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 11:44:49.335296 29272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-2/data/instance:
uuid: "33ce696b438941689204bbeea2e519f6"
format_stamp: "Formatted at 2026-09-26 11:44:49 on dist-test-slave-xxxx"
server_key: "a05710560ecd2e22907d0e0f7f94eb51"
server_key_iv: "947d13f280c41db53bbf681825f38d91"
server_key_version: "encryptionkey@0"
I20260926 11:44:49.335836 29272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-2/wal/instance:
uuid: "33ce696b438941689204bbeea2e519f6"
format_stamp: "Formatted at 2026-09-26 11:44:49 on dist-test-slave-xxxx"
server_key: "a05710560ecd2e22907d0e0f7f94eb51"
server_key_iv: "947d13f280c41db53bbf681825f38d91"
server_key_version: "encryptionkey@0"
I20260926 11:44:49.339527 29272 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260926 11:44:49.341910 29288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 11:44:49.343133 29272 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 11:44:49.343251 29272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-2/data,/tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-2/wal
uuid: "33ce696b438941689204bbeea2e519f6"
format_stamp: "Formatted at 2026-09-26 11:44:49 on dist-test-slave-xxxx"
server_key: "a05710560ecd2e22907d0e0f7f94eb51"
server_key_iv: "947d13f280c41db53bbf681825f38d91"
server_key_version: "encryptionkey@0"
I20260926 11:44:49.343344 29272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 11:44:49.366431 29272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 11:44:49.369644 29272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 11:44:49.369833 29272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 11:44:49.376667 29272 rpc_server.cc:307] RPC server started. Bound to: 127.6.96.60:39353
I20260926 11:44:49.376982 29340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.96.60:39353 every 8 connection(s)
I20260926 11:44:49.377696 29272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-2/data/info.pb
I20260926 11:44:49.380527 29341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 11:44:49.384250 29341 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40039 } has no permanent_uuid. Determining permanent_uuid...
I20260926 11:44:49.387218 6528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBunNuW/build/debug/bin/kudu as pid 29272
I20260926 11:44:49.387336 6528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBunNuW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790422817500949-6528-0/minicluster-data/master-2/wal/instance
I20260926 11:44:49.387605 6528 external_mini_cluster.cc:1428] Setting key 8a7d3a7c24e70408ba57242555bec17b
Sep 26 11:44:49 dist-test-slave-xxxx krb5kdc[29110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423089, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.60@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
Sep 26 11:44:49 dist-test-slave-xxxx krb5kdc[29110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.60@KRBTEST.COM
I20260926 11:44:49.398350 29194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.386153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44446 (local address 127.6.96.62:40039)
0926 11:44:49.386302 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.386306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.387154 (+ 848us) server_negotiation.cc:407] Connection header received
0926 11:44:49.388192 (+ 1038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.388195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.388252 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.388362 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.389764 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.390577 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.391871 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.392146 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.392255 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:49.394929 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:49.394933 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:49.394935 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:49.394965 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:49.396337 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.396833 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.396836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.396837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.396902 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.397243 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.397247 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.397248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.397452 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:49.397545 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:49.398073 (+ 528us) server_negotiation.cc:299] Negotiation successful
0926 11:44:49.398192 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 11:44:49.399374 29343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.386497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.62:40039 (local address 127.0.0.1:44446)
0926 11:44:49.387002 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:49.387052 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:49.387951 (+ 899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:49.388604 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:49.388613 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:49.389020 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:49.389608 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:49.389627 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.390688 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.390690 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:49.391745 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:49.391756 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.392305 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.392308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:49.392481 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:49.393140 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:49.393161 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:49.394707 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:49.396437 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:49.396444 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:49.396454 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:49.396748 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:49.396972 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:49.396975 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:49.396977 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:49.397159 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:49.397549 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:49.397554 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:49.397831 (+ 277us) client_negotiation.cc:769] Sending connection context
0926 11:44:49.398015 (+ 184us) client_negotiation.cc:240] Negotiation successful
0926 11:44:49.398321 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":35,"thread_start_us":130,"threads_started":1}
I20260926 11:44:49.399379 29345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.388268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34810 (local address 127.6.96.60:39353)
0926 11:44:49.388600 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.388610 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.388672 (+ 62us) server_negotiation.cc:407] Connection header received
0926 11:44:49.388777 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.388781 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.388968 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.389094 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.389901 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.391039 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.392062 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.392551 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.392627 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:49.395323 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:49.395329 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:49.395343 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:49.395367 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:49.396972 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.397337 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.397341 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.397342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.397407 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.397654 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.397657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.397659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.397917 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:49.398101 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:49.398292 (+ 191us) server_negotiation.cc:299] Negotiation successful
0926 11:44:49.398405 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":247,"spinlock_wait_cycles":896,"thread_start_us":124,"threads_started":1}
I20260926 11:44:49.401217 29341 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 11:44:49 dist-test-slave-xxxx krb5kdc[29110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423089, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.60@KRBTEST.COM for kudu/127.6.96.61@KRBTEST.COM
Sep 26 11:44:49 dist-test-slave-xxxx krb5kdc[29110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.62@KRBTEST.COM
I20260926 11:44:49.408942 29343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.401657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.61:39523 (local address 127.0.0.1:58900)
0926 11:44:49.401860 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:49.401872 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:49.401939 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:49.402257 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:49.402259 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:49.402377 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:49.402459 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:49.402463 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.403231 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.403234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:49.403753 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:49.403761 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.404192 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.404196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:49.404288 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:49.404780 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:49.404794 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:49.405982 (+ 1188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:49.407588 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:49.407592 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:49.407594 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:49.407798 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:49.408012 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:49.408015 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:49.408017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:49.408099 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:49.408484 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:49.408487 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:49.408592 (+ 105us) client_negotiation.cc:769] Sending connection context
0926 11:44:49.408687 (+ 95us) client_negotiation.cc:240] Negotiation successful
0926 11:44:49.408779 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260926 11:44:49.408974 29271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.401911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58900 (local address 127.6.96.61:39523)
0926 11:44:49.402062 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.402066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.402079 (+ 13us) server_negotiation.cc:407] Connection header received
0926 11:44:49.402116 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.402119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.402160 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.402232 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.402556 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.403138 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.403853 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.404096 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.404182 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:49.406133 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:49.406137 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:49.406140 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:49.406165 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:49.407479 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.407878 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.407881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.407882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.407942 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.408177 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.408180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.408182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.408389 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:49.408481 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:49.408749 (+ 268us) server_negotiation.cc:299] Negotiation successful
0926 11:44:49.408861 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 11:44:49.409519 29266 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } attempt: 3
I20260926 11:44:49.409766 29341 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } has no permanent_uuid. Determining permanent_uuid...
I20260926 11:44:49.409942 29194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.401671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44450 (local address 127.6.96.62:40039)
0926 11:44:49.401773 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.401776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.401786 (+ 10us) server_negotiation.cc:407] Connection header received
0926 11:44:49.401890 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.401893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.401933 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.401995 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.402686 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.403200 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.403910 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.404116 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.404206 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:49.406616 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:49.406619 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:49.406621 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:49.406641 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:49.408084 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.408484 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.408487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.408489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.408571 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.408834 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.408837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.408839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.409087 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:49.409287 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:49.409714 (+ 427us) server_negotiation.cc:299] Negotiation successful
0926 11:44:49.409814 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Sep 26 11:44:49 dist-test-slave-xxxx krb5kdc[29110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423089, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.61@KRBTEST.COM for kudu/127.6.96.60@KRBTEST.COM
Sep 26 11:44:49 dist-test-slave-xxxx krb5kdc[29110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423089, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.60@KRBTEST.COM for kudu/127.6.96.60@KRBTEST.COM
Sep 26 11:44:49 dist-test-slave-xxxx krb5kdc[29110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.96.61@KRBTEST.COM
I20260926 11:44:49.417840 29269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.409849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:39353 (local address 127.0.0.1:34818)
0926 11:44:49.409990 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:49.410029 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:49.410104 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:49.410313 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:49.410316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:49.410455 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:49.410534 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:49.410539 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.411382 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.411386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:49.412047 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:49.412056 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.412603 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.412607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:49.412687 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:49.413132 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:49.413146 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:49.414607 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:49.416355 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:49.416359 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:49.416360 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:49.416619 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:49.416890 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:49.416893 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:49.416895 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:49.416985 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:49.417345 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:49.417347 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:49.417424 (+ 77us) client_negotiation.cc:769] Sending connection context
0926 11:44:49.417550 (+ 126us) client_negotiation.cc:240] Negotiation successful
0926 11:44:49.417678 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260926 11:44:49.417857 29345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.409895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34818 (local address 127.6.96.60:39353)
0926 11:44:49.409992 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.409996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.410070 (+ 74us) server_negotiation.cc:407] Connection header received
0926 11:44:49.410168 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.410171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.410210 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.410616 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.410688 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.411286 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.412148 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.412501 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.412594 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:49.414737 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:49.414741 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:49.414743 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:49.414763 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:49.416261 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.416722 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.416725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.416727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.416797 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.417061 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.417064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.417065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.417255 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:49.417352 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:49.417603 (+ 251us) server_negotiation.cc:299] Negotiation successful
0926 11:44:49.417703 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 11:44:49.419934 29343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.410369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:39353 (local address 127.0.0.1:34820)
0926 11:44:49.410707 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:49.410720 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:49.410781 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:49.411740 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:49.411743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:49.411880 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:49.411953 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:49.411960 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.413144 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.413146 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:49.413835 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:49.413844 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.414246 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.414249 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:49.414323 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:49.414645 (+ 322us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:49.414655 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:49.416469 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:49.418488 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:49.418491 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:49.418493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:49.418725 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:49.418977 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:49.418980 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:49.418982 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:49.419068 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:49.419487 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:49.419490 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:49.419578 (+ 88us) client_negotiation.cc:769] Sending connection context
0926 11:44:49.419667 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 11:44:49.419783 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":1}
I20260926 11:44:49.420378 29346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.410538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34820 (local address 127.6.96.60:39353)
0926 11:44:49.411536 (+ 998us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.411544 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.411555 (+ 11us) server_negotiation.cc:407] Connection header received
0926 11:44:49.411583 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.411586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.411622 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.411700 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.412050 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.413042 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.413929 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.414161 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.414238 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:49.416574 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:49.416578 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:49.416580 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:49.416601 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:49.418385 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.418805 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.418808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.418810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.418904 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.419160 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.419164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.419165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.419390 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:49.419481 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:49.420110 (+ 629us) server_negotiation.cc:299] Negotiation successful
0926 11:44:49.420210 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":928,"spinlock_wait_cycles":3968,"thread_start_us":111,"threads_started":1}
I20260926 11:44:49.420408 29271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.411100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58908 (local address 127.6.96.61:39523)
0926 11:44:49.411198 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.411202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.411214 (+ 12us) server_negotiation.cc:407] Connection header received
0926 11:44:49.411319 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.411322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.411355 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.411413 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.412038 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.412731 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.413534 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.413782 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.413874 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:49.416701 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:49.416705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:49.416707 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:49.416730 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:49.418564 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.419023 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.419026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.419028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.419096 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.419350 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.419353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.419355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.419534 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:49.419758 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:49.419863 (+ 105us) server_negotiation.cc:299] Negotiation successful
0926 11:44:49.419956 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 11:44:49.421271 29266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 099195bf4c0b4fdc9f85853c87a165c1: Bootstrap starting.
I20260926 11:44:49.422950 29191 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } attempt: 3
I20260926 11:44:49.423053 29341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33ce696b438941689204bbeea2e519f6: Bootstrap starting.
I20260926 11:44:49.424057 29266 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 099195bf4c0b4fdc9f85853c87a165c1: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:49.424922 29266 log.cc:824] T 00000000000000000000000000000000 P 099195bf4c0b4fdc9f85853c87a165c1: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:49.425488 29341 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33ce696b438941689204bbeea2e519f6: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:49.426286 29341 log.cc:824] T 00000000000000000000000000000000 P 33ce696b438941689204bbeea2e519f6: Log is configured to *not* fsync() on all Append() calls
I20260926 11:44:49.427462 29266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 099195bf4c0b4fdc9f85853c87a165c1: No bootstrap required, opened a new log
Sep 26 11:44:49 dist-test-slave-xxxx krb5kdc[29110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790423088, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.96.62@KRBTEST.COM for kudu/127.6.96.60@KRBTEST.COM
I20260926 11:44:49.428396 29341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33ce696b438941689204bbeea2e519f6: No bootstrap required, opened a new log
I20260926 11:44:49.430177 6528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 11:44:49.431290 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.423377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.96.60:39353 (local address 127.0.0.1:34824)
0926 11:44:49.423532 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 11:44:49.423547 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 11:44:49.423614 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 11:44:49.423891 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 11:44:49.423894 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 11:44:49.424048 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0926 11:44:49.424139 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:49.424146 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.424897 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.424901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:49.425563 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 11:44:49.425577 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.426099 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.426103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 11:44:49.426184 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:49.426655 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 11:44:49.426671 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 11:44:49.428116 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 11:44:49.429726 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:49.429729 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:49.429731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:49.429969 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:49.430261 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 11:44:49.430264 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 11:44:49.430266 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 11:44:49.430348 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 11:44:49.430776 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 11:44:49.430779 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 11:44:49.430901 (+ 122us) client_negotiation.cc:769] Sending connection context
0926 11:44:49.430998 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 11:44:49.431140 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260926 11:44:49.431779 29346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.423417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34824 (local address 127.6.96.60:39353)
0926 11:44:49.423720 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.423723 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.423731 (+ 8us) server_negotiation.cc:407] Connection header received
0926 11:44:49.423756 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.423757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.423788 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.423845 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.424235 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.424788 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.425698 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.425991 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.426083 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:49.428223 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:49.428226 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:49.428229 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:49.428252 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:49.429631 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.430076 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.430080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.430081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.430164 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.430433 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.430436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.430438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.430648 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:49.430732 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:49.431513 (+ 781us) server_negotiation.cc:299] Negotiation successful
0926 11:44:49.431642 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247}
I20260926 11:44:49.431365 29165 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c5c5b68dd9d447e89dcbfefeaa845525"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44450
I20260926 11:44:49.432227 29341 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33ce696b438941689204bbeea2e519f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40039 } } peers { permanent_uuid: "099195bf4c0b4fdc9f85853c87a165c1" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 } } peers { permanent_uuid: "33ce696b438941689204bbeea2e519f6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } }
I20260926 11:44:49.432462 29341 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33ce696b438941689204bbeea2e519f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33ce696b438941689204bbeea2e519f6, State: Initialized, Role: FOLLOWER
I20260926 11:44:49.432687 29266 raft_consensus.cc:348] T 00000000000000000000000000000000 P 099195bf4c0b4fdc9f85853c87a165c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40039 } } peers { permanent_uuid: "099195bf4c0b4fdc9f85853c87a165c1" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 } } peers { permanent_uuid: "33ce696b438941689204bbeea2e519f6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } }
I20260926 11:44:49.432876 29266 raft_consensus.cc:729] T 00000000000000000000000000000000 P 099195bf4c0b4fdc9f85853c87a165c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 099195bf4c0b4fdc9f85853c87a165c1, State: Initialized, Role: FOLLOWER
I20260926 11:44:49.432819 29341 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33ce696b438941689204bbeea2e519f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40039 } } peers { permanent_uuid: "099195bf4c0b4fdc9f85853c87a165c1" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 } } peers { permanent_uuid: "33ce696b438941689204bbeea2e519f6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } }
I20260926 11:44:49.433166 29266 consensus_queue.cc:260] T 00000000000000000000000000000000 P 099195bf4c0b4fdc9f85853c87a165c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40039 } } peers { permanent_uuid: "099195bf4c0b4fdc9f85853c87a165c1" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 } } peers { permanent_uuid: "33ce696b438941689204bbeea2e519f6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } }
I20260926 11:44:49.433393 29349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33ce696b438941689204bbeea2e519f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40039 } } peers { permanent_uuid: "099195bf4c0b4fdc9f85853c87a165c1" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 } } peers { permanent_uuid: "33ce696b438941689204bbeea2e519f6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } } }
I20260926 11:44:49.433621 29349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33ce696b438941689204bbeea2e519f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:44:49.434216 29341 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33ce696b438941689204bbeea2e519f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:49.434216 29266 sys_catalog.cc:564] T 00000000000000000000000000000000 P 099195bf4c0b4fdc9f85853c87a165c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:49.435134 29350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 099195bf4c0b4fdc9f85853c87a165c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40039 } } peers { permanent_uuid: "099195bf4c0b4fdc9f85853c87a165c1" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 } } peers { permanent_uuid: "33ce696b438941689204bbeea2e519f6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } } }
I20260926 11:44:49.435323 29350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 099195bf4c0b4fdc9f85853c87a165c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:44:49.436064 29191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525: Bootstrap starting.
W20260926 11:44:49.439601 29371 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 33ce696b438941689204bbeea2e519f6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 11:44:49.439663 29371 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 11:44:49.438791 29191 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525: Neither blocks nor log segments found. Creating new log.
I20260926 11:44:49.441352 29191 log.cc:824] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525: Log is configured to *not* fsync() on all Append() calls
W20260926 11:44:49.442749 29372 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 099195bf4c0b4fdc9f85853c87a165c1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 11:44:49.442817 29372 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 11:44:49.440745 29198 master_runner.cc:186] Error getting master registration for 127.6.96.62:40039: OK, instance_id { permanent_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" instance_seqno: 1790423088958256 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 11:44:49.444171 29198 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 11:44:49.444510 29272 master_runner.cc:186] Error getting master registration for 127.6.96.62:40039: OK, instance_id { permanent_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" instance_seqno: 1790423088958256 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 11:44:49.448597 29272 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 11:44:49.448680 29191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525: No bootstrap required, opened a new log
I20260926 11:44:49.451881 29191 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40039 } } peers { permanent_uuid: "099195bf4c0b4fdc9f85853c87a165c1" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 } } peers { permanent_uuid: "33ce696b438941689204bbeea2e519f6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } }
I20260926 11:44:49.452128 29191 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5c5b68dd9d447e89dcbfefeaa845525, State: Initialized, Role: FOLLOWER
I20260926 11:44:49.452481 29191 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40039 } } peers { permanent_uuid: "099195bf4c0b4fdc9f85853c87a165c1" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 } } peers { permanent_uuid: "33ce696b438941689204bbeea2e519f6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } }
I20260926 11:44:49.453030 29376 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40039 } } peers { permanent_uuid: "099195bf4c0b4fdc9f85853c87a165c1" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 } } peers { permanent_uuid: "33ce696b438941689204bbeea2e519f6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } } }
I20260926 11:44:49.453168 29376 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:44:49.453913 29191 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 11:44:49.453963 29165 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 11:44:49.454077 29165 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:49.455183 29165 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40039 } } peers { permanent_uuid: "099195bf4c0b4fdc9f85853c87a165c1" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 } } peers { permanent_uuid: "33ce696b438941689204bbeea2e519f6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } }
I20260926 11:44:49.456027 29165 leader_election.cc:288] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525 [CANDIDATE]: Term 1 election: Requested vote from peers 099195bf4c0b4fdc9f85853c87a165c1 (127.6.96.61:39523), 33ce696b438941689204bbeea2e519f6 (127.6.96.60:39353)
W20260926 11:44:49.456310 29144 tablet.cc:2369] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 11:44:49.456956 29315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "33ce696b438941689204bbeea2e519f6"
I20260926 11:44:49.457190 29315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33ce696b438941689204bbeea2e519f6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:49.458036 29240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "099195bf4c0b4fdc9f85853c87a165c1"
I20260926 11:44:49.458288 29240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 099195bf4c0b4fdc9f85853c87a165c1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 11:44:49.458465 29315 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 33ce696b438941689204bbeea2e519f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5c5b68dd9d447e89dcbfefeaa845525 in term 1.
I20260926 11:44:49.459046 29142 leader_election.cc:302] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33ce696b438941689204bbeea2e519f6, c5c5b68dd9d447e89dcbfefeaa845525; no voters:
I20260926 11:44:49.459661 29240 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 099195bf4c0b4fdc9f85853c87a165c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5c5b68dd9d447e89dcbfefeaa845525 in term 1.
I20260926 11:44:49.459903 29376 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 11:44:49.460122 29376 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525 [term 1 LEADER]: Becoming Leader. State: Replica: c5c5b68dd9d447e89dcbfefeaa845525, State: Running, Role: LEADER
I20260926 11:44:49.460410 29376 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40039 } } peers { permanent_uuid: "099195bf4c0b4fdc9f85853c87a165c1" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 } } peers { permanent_uuid: "33ce696b438941689204bbeea2e519f6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } }
I20260926 11:44:49.461912 29376 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5c5b68dd9d447e89dcbfefeaa845525. Latest consensus state: current_term: 1 leader_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40039 } } peers { permanent_uuid: "099195bf4c0b4fdc9f85853c87a165c1" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 } } peers { permanent_uuid: "33ce696b438941689204bbeea2e519f6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } } }
I20260926 11:44:49.462007 29376 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:49.463478 29388 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 11:44:49.466730 29388 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 11:44:49.478987 29194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.457970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44460 (local address 127.6.96.62:40039)
0926 11:44:49.458088 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.458093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.469168 (+ 11075us) server_negotiation.cc:407] Connection header received
0926 11:44:49.469370 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.469374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.469425 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.469477 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.470635 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.471525 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.472876 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.473230 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.473306 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:49.475458 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:49.475463 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:49.475466 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:49.475495 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:49.477078 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.477572 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.477576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.477578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.477665 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.477957 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.477960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.477962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.478183 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:49.478292 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:49.478677 (+ 385us) server_negotiation.cc:299] Negotiation successful
0926 11:44:49.478799 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 11:44:49.487138 29315 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 33ce696b438941689204bbeea2e519f6 [term 1 FOLLOWER]: Refusing update from remote peer c5c5b68dd9d447e89dcbfefeaa845525: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:44:49.487727 29346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.458669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34832 (local address 127.6.96.60:39353)
0926 11:44:49.458872 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.458877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.471262 (+ 12385us) server_negotiation.cc:407] Connection header received
0926 11:44:49.471449 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.471453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.471505 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.471553 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.472494 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.473292 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.474422 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.474776 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.474875 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:49.483712 (+ 8837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:49.483716 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:49.483719 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:49.483749 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:49.485481 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.486383 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.486387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.486388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.486491 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.486773 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.486776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.486778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.487007 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:49.487105 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:49.487462 (+ 357us) server_negotiation.cc:299] Negotiation successful
0926 11:44:49.487584 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260926 11:44:49.487960 29386 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33ce696b438941689204bbeea2e519f6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:44:49.489701 29271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:49.463218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58916 (local address 127.6.96.61:39523)
0926 11:44:49.463339 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.463343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.476383 (+ 13040us) server_negotiation.cc:407] Connection header received
0926 11:44:49.480591 (+ 4208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.480594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.480652 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.480706 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.481471 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.482241 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.483416 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:49.483765 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:49.483834 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:49.485318 (+ 1484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 11:44:49.485322 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 11:44:49.485325 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 11:44:49.485352 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 11:44:49.486874 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.487315 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.487320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.487322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.487404 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 11:44:49.487692 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 11:44:49.487695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 11:44:49.487697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 11:44:49.487903 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 11:44:49.487991 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:49.489288 (+ 1297us) server_negotiation.cc:299] Negotiation successful
0926 11:44:49.489407 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 11:44:49.492784 29349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33ce696b438941689204bbeea2e519f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5c5b68dd9d447e89dcbfefeaa845525. Latest consensus state: current_term: 1 leader_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40039 } } peers { permanent_uuid: "099195bf4c0b4fdc9f85853c87a165c1" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 } } peers { permanent_uuid: "33ce696b438941689204bbeea2e519f6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } } }
I20260926 11:44:49.492916 29349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33ce696b438941689204bbeea2e519f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:44:49.493997 29240 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 099195bf4c0b4fdc9f85853c87a165c1 [term 1 FOLLOWER]: Refusing update from remote peer c5c5b68dd9d447e89dcbfefeaa845525: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 11:44:49.494614 29386 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525 [LEADER]: Connected to new peer: Peer: permanent_uuid: "099195bf4c0b4fdc9f85853c87a165c1" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 11:44:49.498302 29350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 099195bf4c0b4fdc9f85853c87a165c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5c5b68dd9d447e89dcbfefeaa845525. Latest consensus state: current_term: 1 leader_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40039 } } peers { permanent_uuid: "099195bf4c0b4fdc9f85853c87a165c1" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 } } peers { permanent_uuid: "33ce696b438941689204bbeea2e519f6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } } }
I20260926 11:44:49.498443 29350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 099195bf4c0b4fdc9f85853c87a165c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:44:49.505649 29386 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40039 } } peers { permanent_uuid: "099195bf4c0b4fdc9f85853c87a165c1" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 } } peers { permanent_uuid: "33ce696b438941689204bbeea2e519f6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } } }
I20260926 11:44:49.505798 29386 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:49.506492 29376 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40039 } } peers { permanent_uuid: "099195bf4c0b4fdc9f85853c87a165c1" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 } } peers { permanent_uuid: "33ce696b438941689204bbeea2e519f6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } } }
I20260926 11:44:49.506610 29376 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525 [sys.catalog]: This master's current role is: LEADER
I20260926 11:44:49.507717 29400 mvcc.cc:205] Tried to move back new op lower bound from 7333572974529867776 to 7333572974432759808. Current Snapshot: MvccSnapshot[applied={T|T < 7333572974529867776}]
I20260926 11:44:49.507870 29349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33ce696b438941689204bbeea2e519f6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40039 } } peers { permanent_uuid: "099195bf4c0b4fdc9f85853c87a165c1" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 } } peers { permanent_uuid: "33ce696b438941689204bbeea2e519f6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } } }
I20260926 11:44:49.507983 29349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33ce696b438941689204bbeea2e519f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:44:49.509287 29350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 099195bf4c0b4fdc9f85853c87a165c1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5c5b68dd9d447e89dcbfefeaa845525" member_type: VOTER last_known_addr { host: "127.6.96.62" port: 40039 } } peers { permanent_uuid: "099195bf4c0b4fdc9f85853c87a165c1" member_type: VOTER last_known_addr { host: "127.6.96.61" port: 39523 } } peers { permanent_uuid: "33ce696b438941689204bbeea2e519f6" member_type: VOTER last_known_addr { host: "127.6.96.60" port: 39353 } } }
I20260926 11:44:49.509408 29350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 099195bf4c0b4fdc9f85853c87a165c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 11:44:49.510106 29402 mvcc.cc:205] Tried to move back new op lower bound from 7333572974529867776 to 7333572974432759808. Current Snapshot: MvccSnapshot[applied={T|T < 7333572974529867776}]
I20260926 11:44:49.510264 29399 mvcc.cc:205] Tried to move back new op lower bound from 7333572974529867776 to 7333572974432759808. Current Snapshot: MvccSnapshot[applied={T|T < 7333572974529867776}]
I20260926 11:44:49.518756 29388 catalog_manager.cc:1347] Generated new cluster ID: 5496f9db4dbf4413b4863a3a381ec485
I20260926 11:44:49.518874 29388 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 11:44:49.541091 29388 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 11:44:49.542531 29388 catalog_manager.cc:1504] Loading token signing keys...
I20260926 11:44:49.571769 29388 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5c5b68dd9d447e89dcbfefeaa845525: Generated new TSK 0
I20260926 11:44:49.572737 29388 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 11:44:49.602317 29194 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:49.599594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44472 (local address 127.6.96.62:40039)
0926 11:44:49.599741 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.599746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.600972 (+ 1226us) server_negotiation.cc:407] Connection header received
0926 11:44:49.601411 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.601415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.601488 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.601550 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.602183 (+ 633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44472: BlockingRecv error: recv got EOF from 127.0.0.1:44472 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260926 11:44:49.603508 29346 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:49.600359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34840 (local address 127.6.96.60:39353)
0926 11:44:49.600469 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.600474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.601318 (+ 844us) server_negotiation.cc:407] Connection header received
0926 11:44:49.602693 (+ 1375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.602696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.602746 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.602798 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.603371 (+ 573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34840: BlockingRecv error: recv got EOF from 127.0.0.1:34840 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260926 11:44:49.605558 29271 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:49.604324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58930 (local address 127.6.96.61:39523)
0926 11:44:49.604456 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.604460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.604674 (+ 214us) server_negotiation.cc:407] Connection header received
0926 11:44:49.604850 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.604854 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.604903 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.604955 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.605442 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58930: BlockingRecv error: recv got EOF from 127.0.0.1:58930 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260926 11:44:49.610059 29271 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:49.608498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58946 (local address 127.6.96.61:39523)
0926 11:44:49.608598 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.608602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.609087 (+ 485us) server_negotiation.cc:407] Connection header received
0926 11:44:49.609306 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.609310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.609387 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.609456 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.609955 (+ 499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58946: BlockingRecv error: recv got EOF from 127.0.0.1:58946 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260926 11:44:49.615392 29271 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:49.613764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58962 (local address 127.6.96.61:39523)
0926 11:44:49.613879 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.613883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.614377 (+ 494us) server_negotiation.cc:407] Connection header received
0926 11:44:49.614598 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.614601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.614657 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.614711 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.615277 (+ 566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58962: BlockingRecv error: recv got EOF from 127.0.0.1:58962 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260926 11:44:49.621308 29271 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:49.619763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58966 (local address 127.6.96.61:39523)
0926 11:44:49.619875 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.619879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.620371 (+ 492us) server_negotiation.cc:407] Connection header received
0926 11:44:49.620575 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.620578 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.620629 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.620682 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.621203 (+ 521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58966: BlockingRecv error: recv got EOF from 127.0.0.1:58966 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260926 11:44:49.629206 29271 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:49.627619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58970 (local address 127.6.96.61:39523)
0926 11:44:49.627757 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.627761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.628245 (+ 484us) server_negotiation.cc:407] Connection header received
0926 11:44:49.628440 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.628444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.628495 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.628549 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.629090 (+ 541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58970: BlockingRecv error: recv got EOF from 127.0.0.1:58970 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260926 11:44:49.646252 29271 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:49.639713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58972 (local address 127.6.96.61:39523)
0926 11:44:49.639849 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.639855 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.645361 (+ 5506us) server_negotiation.cc:407] Connection header received
0926 11:44:49.645488 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.645492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.645549 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.645628 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.646133 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58972: BlockingRecv error: recv got EOF from 127.0.0.1:58972 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260926 11:44:49.665688 29271 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:49.664421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58984 (local address 127.6.96.61:39523)
0926 11:44:49.664548 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.664552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.664847 (+ 295us) server_negotiation.cc:407] Connection header received
0926 11:44:49.664984 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.664988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.665040 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.665108 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.665564 (+ 456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58984: BlockingRecv error: recv got EOF from 127.0.0.1:58984 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260926 11:44:49.700925 29271 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:49.699759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59000 (local address 127.6.96.61:39523)
0926 11:44:49.699927 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.699931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.700084 (+ 153us) server_negotiation.cc:407] Connection header received
0926 11:44:49.700151 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.700154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.700204 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.700292 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.700766 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59000: BlockingRecv error: recv got EOF from 127.0.0.1:59000 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260926 11:44:49.768070 29271 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:49.766909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59006 (local address 127.6.96.61:39523)
0926 11:44:49.767058 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.767063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.767188 (+ 125us) server_negotiation.cc:407] Connection header received
0926 11:44:49.767339 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.767343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.767394 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.767461 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.767948 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59006: BlockingRecv error: recv got EOF from 127.0.0.1:59006 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260926 11:44:49.899164 29271 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:49.898029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59008 (local address 127.6.96.61:39523)
0926 11:44:49.898186 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:49.898191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:49.898295 (+ 104us) server_negotiation.cc:407] Connection header received
0926 11:44:49.898442 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:49.898445 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:49.898493 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:49.898563 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:49.899040 (+ 477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59008: BlockingRecv error: recv got EOF from 127.0.0.1:59008 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260926 11:44:50.158422 29271 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 11:44:50.157253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59016 (local address 127.6.96.61:39523)
0926 11:44:50.157395 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:50.157399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:50.157459 (+ 60us) server_negotiation.cc:407] Connection header received
0926 11:44:50.157660 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:50.157664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:50.157716 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:50.157796 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 11:44:50.158299 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59016: BlockingRecv error: recv got EOF from 127.0.0.1:59016 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 11:44:50.441011 29371 catalog_manager.cc:1259] Loaded cluster ID: 5496f9db4dbf4413b4863a3a381ec485
I20260926 11:44:50.441088 29371 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 33ce696b438941689204bbeea2e519f6: loading cluster ID for follower catalog manager: success
I20260926 11:44:50.442871 29371 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 33ce696b438941689204bbeea2e519f6: acquiring CA information for follower catalog manager: success
I20260926 11:44:50.443938 29372 catalog_manager.cc:1259] Loaded cluster ID: 5496f9db4dbf4413b4863a3a381ec485
I20260926 11:44:50.443997 29372 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 099195bf4c0b4fdc9f85853c87a165c1: loading cluster ID for follower catalog manager: success
I20260926 11:44:50.444041 29371 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 33ce696b438941689204bbeea2e519f6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 11:44:50.445514 29372 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 099195bf4c0b4fdc9f85853c87a165c1: acquiring CA information for follower catalog manager: success
I20260926 11:44:50.446600 29372 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 099195bf4c0b4fdc9f85853c87a165c1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 11:44:50.676155 29471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:50.672487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59032 (local address 127.6.96.61:39523)
0926 11:44:50.672700 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:50.672704 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:50.672722 (+ 18us) server_negotiation.cc:407] Connection header received
0926 11:44:50.672857 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:50.672860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:50.672983 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:50.673095 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:44:50.673377 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:50.673905 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:50.674794 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:50.675046 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:50.675120 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:50.675388 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:44:50.675821 (+ 433us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:44:50.675913 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:50.675973 (+ 60us) server_negotiation.cc:299] Negotiation successful
0926 11:44:50.676025 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":62,"threads_started":1}
I20260926 11:44:50.682722 29477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:50.679058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34842 (local address 127.6.96.60:39353)
0926 11:44:50.679273 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:50.679277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:50.679289 (+ 12us) server_negotiation.cc:407] Connection header received
0926 11:44:50.679434 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:50.679437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:50.679552 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:50.679646 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:44:50.679829 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:50.680339 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:50.681220 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:50.681467 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:50.681547 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:50.681883 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:44:50.682330 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:44:50.682434 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:50.682528 (+ 94us) server_negotiation.cc:299] Negotiation successful
0926 11:44:50.682595 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":67,"threads_started":1}
I20260926 11:44:50.689858 29471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:50.685693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59044 (local address 127.6.96.61:39523)
0926 11:44:50.685837 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:50.685842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:50.686097 (+ 255us) server_negotiation.cc:407] Connection header received
0926 11:44:50.686288 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:50.686292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:50.686350 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:50.686428 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:44:50.686677 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:50.687496 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:50.688748 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:50.689045 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:50.689135 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:50.689399 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:44:50.689469 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:44:50.689530 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:50.689653 (+ 123us) server_negotiation.cc:299] Negotiation successful
0926 11:44:50.689714 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 11:44:50.689913 29484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:50.685627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44480 (local address 127.6.96.62:40039)
0926 11:44:50.685879 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:50.685883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:50.686103 (+ 220us) server_negotiation.cc:407] Connection header received
0926 11:44:50.686202 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:50.686205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:50.686363 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:50.686469 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:44:50.686812 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:50.687601 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:50.688496 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:50.688785 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:50.688889 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:50.689200 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:44:50.689554 (+ 354us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:44:50.689633 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:50.689706 (+ 73us) server_negotiation.cc:299] Negotiation successful
0926 11:44:50.689763 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":70,"threads_started":1}
I20260926 11:44:50.690943 29477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 11:44:50.686018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34854 (local address 127.6.96.60:39353)
0926 11:44:50.686128 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0926 11:44:50.686132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 11:44:50.686363 (+ 231us) server_negotiation.cc:407] Connection header received
0926 11:44:50.686476 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 11:44:50.686479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 11:44:50.686530 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 11:44:50.686599 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 11:44:50.687730 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:50.688533 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:50.689598 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 11:44:50.689885 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 11:44:50.690130 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 11:44:50.690372 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 11:44:50.690446 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 11:44:50.690522 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 11:44:50.690720 (+ 198us) server_negotiation.cc:299] Negotiation successful
0926 11:44:50.690780 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 11:44:50.693302 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 29123
I20260926 11:44:50.698748 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 29198
I20260926 11:44:50.703797 6528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBunNuW/build/debug/bin/kudu with pid 29272
2026-09-26T11:44:50Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2997 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6020 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (273196 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST